OPF Memory Usage

Karl Schendel wrote:
> 
> In article <19970301232201.SAA28293@ladder02.news.aol.com>,
> priyeshp@aol.com (PRIYESHP) wrote:
> 
> > I am trying to determine what effect a high value for opf memory has on
> > actual
> > memory usage....
> >
> > This is what I am guessing :
> >
> > When a thread enters the optimiser, the dbms process will grab memory
> > equivalent to (opf.memory / opf.active). If opf.memory is defined as
> > 10,000,000 and opf.active 100 then the amount of memory used
> > by each thread when in the optimiser will be 100K. If two threads are in
> > the
> > optimiser then the amount of memory used is 200K. When the thread leaves
> > the optimiser this memory is released.
> >
> > Question is, is the above true ?
> 
> While we really need Doug Inkster or someone else familiar with OPF internals
> to answer this, my guess is that the OPF memory pool is allocated in
> its entirety at startup.  Whether a thread can grab all of it, or
> is forcibly limited to opf.memory/opf.active, I wouldn't venture to
> guess.  But historically the iidbms does not seem to have been very big
> on dynamic allocation (meaning creation) of its memory pools.
> 
> I have snooped the iidbms periodically, but for some reason memory allocation
> was never one of the things I concentrated on.  Perhaps because it seemed
> too depressing a subject... :-)
> 
> Karl

The truth, as always, lies in between. It is indeed true that the memory
available per OPF thread is opf.memory/opf.active. On termination of a
thread, the memory is NOT released back to the OS. It is, however,
released to a pool of memory available for allocation to future OPF
threads (the OPF pool). And opf.memory is NOT allocated in one big chunk
at server startup time. Individual OPF sessions are incrementally
allocated memory as they request it. The upper bound for a session is
the opf.memory/opf.active, mentioned above. But if a session has low
memory requirements (e.g. to compile a simple single table select), it
is only allocated what it needs. Assuming a session has not exceeded the
upper bound, an attempt is first made to satisfy the memory request from
the OPF pool (i.e., OPF memory allocated, then freed by now-terminated
sessions). If enough memory is not available in the existing OPF pool, a
request for another "chunk" of memory is made from the OS (in Unix we
use the sbrk function to peel the chunk off the end of the allocated
address space), the chunk is added to the OPF pool and the request is
completed. So both at the session level and the server level, memory is
added incrementally as needed up to the defined limits.

Now for the 2.0 sneak preview. 2.0 has a configurable parameter which
gives control over the proportion of opf.memory which can be consumed by
any one session. This will allow the OPF memory pool to be configured
for the "average" query, rather than the "boundary" query. Those
relatively few queries with higher memory requirements will be able to
exceed the previous opf.memory/opf.active limit, on the assumption that
it would be rare to have enough such queries compiling concurrently to
cause the OPF pool to be exhausted.

Hope this helps.

Doug.



On Mar 4,  9:37, Doug Inkster wrote:
> Subject: Re: OPF Memory usage
> ..... 2.0 has a configurable parameter which
> gives control over the proportion of opf.memory which can be consumed by
> any one session. This will allow the OPF memory pool to be configured
> for the "average" query, rather than the "boundary" query.

An excellent idea, provided that one has some method (IMA?? or something)
to determine how much OPF memory said "average" query uses.
No doubt such a thing exists, or can be made to exist?

I'm glad that my impression of how OPF uses memory was pessimistic.
Back in the early days of the version 6 server, one got the impression
(rightly or wrongly) that the server was incredibly obtuse in its memory
allocation techniques.  Well, maybe not obtuse, let's say inflexible.
It would seem that progress had been made, thank goodness.


-- 
Karl Schendel            Phone: (412) 963-8844
Telesis Computer Corp      Fax: (412) 963-1373
wiz@telesismfg.com
Ian Britten wrote:
Ingres Q & A
To William's Home Page
Email William