jointprd@crl.com (John Bishop) writes:
: Before I begin, please don't flame me to badly.
:
: I'm a Sybase DBA (with 2 yrs of Ingres exp); I've got a friend who's
: an Ingres DBA. We were discussing the merits of raw partitions
: vs. file systems in a RDBMS environment.
:
: Sybase strongly recommends that raw partitions be used (due to cacheing
: issues that files systems have). However, file systems tend to
: perform better (again, due to cacheing issues). The $100,000
: question is where does Ingres (or you) stand on this issue?
:
: My stance is that if the Server goes down, and a "committed" transaction
: is still in cache (due to the way Unix handles writes), you will
: lose the transaction and the database is now corrupt.
Fortunately that is not how Ingres works. With Ingres you have two
choices: fast-commit, or not. If you choose to use fast-commit,
transactions are logged in the transaction log file synchronously and
without buffering. (And the log file can be a raw device if you like.)
The updates are then written to the underlying database files when
Ingres and the operating system "feel like it." In the event of a
crash, the recovery process that starts up when the system is restarted
looks in the transaction log and makes sure that all the transactions
it finds there got posted to the database files. This is the so-called
"redo". This process also backs out uncommitted transactions that got
posted to the underlying files, which is the converse of the problem you
pose. Users are only allowed back into the database after the recovery
process finishes.
The other alternative is not to use fast-commit. In that case updates
are posted to the files synchronously and without buffering, even
though the files reside in an operating system managed file system. ie
the Unix buffering/cacheing is disabled. The performance of this
approach is not great though.
I think most sites opt to use fast-commit BTW, and we have very little
trouble with it. I could go on about this, but we quickly get into
deep water, so unless you are really interested I won't.
--Roy Hann
Subject: Re: Meanings of fast-commit.
Date: 29 Aug 1995 04:23:55 GMT
In article hideyasu@kaka.cse.canon.co.jp (Hideyasu Suzuki) writes:
>Hello, Ingres Expert:
>
>I have a question about fast-commit on Ingres 6.4/04. My question is
>the timing that the modified data pages are written to disk using the
>fast-commit.
>
>I've thought that no modified data pages are written to disk until
>commit issues, but my friend told me that, using fast-commit, modified
>data pages are written to disk when the log file reach to consistency
>point even if there is no commit issued.
>
>Which is correct??? Or, both of them are incorrect???
>
>Thanx in advance.
>--
>---
> Hideyasu Suzuki
> Canon Inc, Shin Kawasaki
>
Modified data pages are written to disk when
1) WBSTART is reached, by the write-behind thread(s)
2) A consistency point is performed, by the fast-commit thread
Modified data pages are NOT written to disk by commit. Log buffers
are written to the log file, but not data pages. This is why it is
called "fast commit". If the server should fail, the log file contains
the information needed to "redo" the transaction.
Cheers,
|--------------------------------------------------------------------------|
| Michael Leo | The Ingres FAQ is at ftp.adc.com, /pub/ingres. |
| York & Associates, Inc.| Also check out /pub/ingres/utilities/NAIUA for |
| Minneapolis, MN, USA | the NAIUA Tool Kit. Lastly, access all this via|
| (612) 921-8083 (voice) | WWW at http://www.adc.com/ingres/ing-top.html. |
| mal@winternet.com | All constructive suggestions/criticism welcome. |
|--------------------------------------------------------------------------|
In article <41vgda$qge@qvarsx.er.usgs.gov> schlesin@dg0daztcn.wr.usgs.gov (Mark Schlesinger) writes:
>
>In article <41totq$qom@agate.berkeley.edu>, pbrown@triplerock.CS.Berkeley.EDU (Paul Brown) writes:
>> Mark Schlesinger (schlesin@dg0daztcn.wr.usgs.gov) wrote:
>>
>> : The first one is correct and the second one is wrong. There can be
>> : NO consistency point while a transaction is still open. So the pages can
>> : NOT be written to disk until a commit is issued.
>>
>> OK. I'll bite.
>>
>> I have a table which is 100Meg in size.
>>
>> I have a 200M log file.
>>
>> I have a memory cache which is 64Meg in size.
>>
>> I update every row in the table. I change every page.
>>
>> If the pages aren't written to disk before the transaction's
>> completed, where do they go?
>>
> They go to the log buffers, then to the logfile until the
>log file fills up and gives you an error or the transaction is committed
>and the data is written to the table.
>
>--
>Mark Schlesinger If I had any opinions...They'd be mine!!!
>schlesin@maildaztcn.wr.usgs.gov
Pages from the DMF cache are NEVER written to the log buffer.
Remember, even uncommitted transactions can cause changes to the REAL
database tables. This happens when the DMF cache is flushed, either
entirely or partially by a write-behind wake-up (WBSTART is reached).
If the transaction is not committed, the log file data is used to
"repair" any writes done to the real database files.
Cheers,
|--------------------------------------------------------------------------|
| Michael Leo | The Ingres FAQ is at ftp.adc.com, /pub/ingres. |
| York & Associates, Inc.| Also check out /pub/ingres/utilities/NAIUA for |
| Minneapolis, MN, USA | the NAIUA Tool Kit. Lastly, access all this via|
| (612) 921-8083 (voice) | WWW at http://www.adc.com/ingres/ing-top.html. |
| mal@winternet.com | All constructive suggestions/criticism welcome. |
|--------------------------------------------------------------------------|
Paul Brown (pbrown@triplerock.CS.Berkeley.EDU) wrote:
: So, on commit, Ingres reads all these pages out of the log file and
: back into the disk files?
No, That is done by the DBMS Server(s), out of the DMF cache. Either when
the wbstart threshold is reached, or when a consistency point is reached.(You
could write a small chapter for the FAQ on the workings of the various
internal threads. I wont try and go into it in one paragraph.)
If at this time the DBMS server should fail, the *recovery process*
(dmfrcp) would have to read the log file and write to the disk files.
With 'normal' operation the dbms server would not read the log file again,
it would merely write the modified pages out from the cache to the disk
files.
: So when I update my 1000 pages, I get to write it twice and read it
: twice. (File -> DMF, DMF -> LogFile, LogFile -> DMF, DMF -> File).
: Whew. That's a *lot* of work. ;-)
It would be, *IF* that was what ingres does, but it is a bit smarter than
that.
If you know all of this, and was merely making a sarcastic remark, please
ignore this posting! ;-)
Regards
--
Cor
The Data Base Approach cc _______ Cor Winckler
PO Box 5165, Helderberg |\ | Internet: cor@dba.co.za
7135 Somerset West | >=====| Tel: 0(+27) 21 557 5404
South Africa |/______| Fax: 0(+27) 21 881 3318
On Aug 29, 22:11, Paul Brown wrote:
> So, on commit, Ingres reads all these pages out of the log file and
> back into the disk files?
>
> So when I update my 1000 pages, I get to write it twice and read it
> twice. (File -> DMF, DMF -> LogFile, LogFile -> DMF, DMF -> File).
No, no. Fast-commit simply implies that at commit, pages MAY not have
been written from the buffer cache to the database; but the log records
WILL have been written to the log. If a page isn't written to the db
at commit time, it simply gets written later on when the buffer is
needed, or when write-behind gets it, or when a consistency point
writes all dirty buffers.
the only time the logfile needs to be re-read is if the system crashes
after a commit but before a page is written to the db. (actually,
before a consistency point, i think). recovery will read the log and
redo all committed transactions since the last CP in case the
pages in question didn't get written to the db.
without fast commit, all pages dirtied by a transaction have to be flushed
at commit time, which stalls the session at the commit waiting for
writes to complete.
--
Karl Schendel Phone: (412) 963-8844
Telesis Computer Corp Fax: (412) 963-1373
wiz@telesis.com
In article mal@winternet.com (Michael Leo) writes:
>From: mal@winternet.com (Michael Leo)
>Subject: Re: Fast-Commit & Write Behinds
>Date: 09 Nov 1995 19:32:25 GMT
>In article martinm@planet.mh.dpi.qld.gov.au (Leyland Scar) writes:
>>
>>Question?
>>
>>When the fast commit flushes is < than the write behind flushes, does this
>>indicate the the system swings between wbstart and wend limits of the cache
>>causing the system to continually stop and then startup the write behind
>>sessions (eg system becomes "jerky"). The fast commit flushes is < than the
>>write behind flushes for the first two DBMS servers as stated in the dm420
>>output.
>>
>>Do I need to change the dmf cache parameters to "smooth" out the write behind
>>flushes for these two servers?
>The "fast commit flushes" are consistency points. You generally need
>to keep those lower than "write behind flushes". But maybe not.
>Depends upon what you are trying to accomplish. If you never expect to
>take advantage of the write-behind caching (i.e. you don't update the
>same page multiple times), you should tune the WB parms to wake up often
>and keep the number of modified pages low. Then the rest of the
>cache is used for a "read" cache. CPs will be quick and painless.
>However, if you constantly update the same pages over and over by different
>users, you may need to choose between the "jerkiness" of a CP and the
>cost of writing the page via write-behind threads.
>Generally, I try to tune the write-behind threads to "ease the pain" of a
>CP. When the inevitable CP occurs, it has little work to do. However,
>this doesn't promote a high hit rate on the dirty pages, as they aren't
>dirty very long.
Thanks for that Michael!
Lets say that you have two sanarios;
1) dmf.cache_size 5000
dmf.start 1500
dmf.end 500
2) dmf.cache_size 5000
dmf.start 400
dmf.end 200
With sanario 1, the cache would tend to have more dirty pages than sanario 2
because of the larger dmf.start etc. So when a CP comes along, more i/o work
has to be performed because of the larger number of pages it has to write to
disk. The hit-rate% would be good for sanario 1 but bad for sanario 2.
Yep. The trade-off would be hit-rate% and i/o work performed!
Do fast-commit flushes actually flush the cache or just write committed
transactions to the database?
Michael Martin,
DBA, DPI,QLD, Austr.
-
Ingres Q & A
-
To William's Home Page
© William Yuan 2000
Email William