Large Pages on AIX

How do you set up large pages on AIX?

To determine which large page size (64 KB, 16 MB, or 16 GB) is supported on your system:

pagesize -af

64 KB large pages are enabled by default (if they are supported on your system):

vmo -L vmm_mpsize_support

To enable 16 MB large pages on AIX (if they are supported on your system):

vmo -r -o lgpg_regions= -o lgpg_size=16777216 shutdown -Fr

To enable 16 GB large pages:

- On the managed system, go to:

Properties
Memory
Advanced Options
Show Details to change the number of 16 GB pages.

Assign 16 GB huge pages to a partition by changing the partition's profile.

Go Back to the Top

How do I monitor the usage of large pages on AIX?

To monitor memory pools and large pages:


svmon

If 'PoolSize' decreases, run:

kdb

memp * //NUMFRB is the number of frames in the specific memory pool.

vmpool -l * //LGPG_FREE is the amount of free large pages.

q // To quit out of kdb.

tprof -a -y -O all //used for large page analysis

*Note: If you specify the -y flag, only the specified program and its descendents are profiled. Use the -y (Turns on the event-based profiling for only the specified command and its descendents) flag only with the -E ( - Enables event-based profiling) or -a (Turns on the large page analysis) flag.

To determine if large pages are NOT being used:

ipcs -mP //Make note of the 'LGPG: -' field

vmstat -l //Make note of the 'alp' (Active Large Pages) column

svmon //Make note of the 'PageSize' and 'inuse' column.


Go Back to the Top

How do I make my application use the large pages?

Run the following commands to use large pages:

    chuser capabilities=CAP_BYPASS_RAC_VMM, CAP_PROPAGATE lsuser -a capabilities vmstat -l

*Note: alp: Active Large Pages flp: Free Large Pages


as data and heap segments:

run ONE of the following commands:

ldedit -blpdata

OR

LDR_CNTRL=LARGE_PAGE=Y

export LDR_CNTRL=LARGE_PAGE=Y@LARGE_PAGE_DATA=M

as shared memory:

Modify the application to request large pages for shared memory by:

    specifying an appropriate value for the "size" parameter passing the SHM_PAGE and SHM_PIN flags to the shmget() system call. running 'vmo -o v_pinshm=1' (which is a dynamic change, so NO system reboot is needed).

To see if the application is using large pages:

truss -d -f -o /tmp/truss.out -s 16777216 -p
Go Back to the Top

Why doesn't the number of large pages remain static (or stay the same)?

The 'PoolSize' for large pages (from svmon output) is NOT supposed to remain static (or stay the same) if:

- the system cannot fulfill a memory request from the 4 KB pages. VMM (Virtual Memory Manager) then:

    searches for available large pages, converts them back to 4 KB pages, and fulfills the memory request.

the system cannot fulfill a memory request from the 4K pages if memory pools are unbalanced. VMM then:

    searches for available large pages, converts them back to 4 KB pages, and fulfills the memory request.

*Note: Once those large pages are converted to 4K, they do NOT get converted back to large pages (of a 64 KB, 16 MB, or 16 GB size) until the system is rebooted. So, this is working as designed. An error report will get created when converting a large page to 4 KB page:


LABEL: LGPG_FREED

IDENTIFIER: C4C3339D

Date/Time: Tue Oct 14 18:53:44 CDT 2008

Sequence Number: 3349

Machine Id: 00C517B04C00

Node Id: medsg-min-prod-midr-1

Class: S

Type: INFO

Resource Name: SYSVMM

Description

ONE OR MORE LARGE PAGES HAS BEEN CONVERTED INTO PAGEABLE PAGES

Probable Causes

System at or near pinned memory limit.

Recommended Actions

Tune maxpin percentage or lgpg_regions.

Detail Data

Number of large pages attempted to free:

1

Number of large pages actually freed:

Go Back to the Top