- >>> cat /proc/meminfo
- MemTotal: 16345480 kB
- MemFree: 1770128 kB
- Buffers: 382184 kB
- Cached: 10432632 kB
- SwapCached: 0 kB
- Active: 9228324 kB
- Inactive: 4621264 kB
- Active(anon): 7019996 kB
- Inactive(anon): 548528 kB
- Active(file): 2208328 kB
- Inactive(file): 4072736 kB
- Unevictable: 0 kB
- Mlocked: 0 kB
- SwapTotal: 0 kB
- SwapFree: 0 kB
- Dirty: 3432 kB
- Writeback: 0 kB
- AnonPages: 3034588 kB
- Mapped: 4243720 kB
- Shmem: 4533752 kB
- Slab: 481728 kB
- SReclaimable: 440712 kB
- SUnreclaim: 41016 kB
- KernelStack: 1776 kB
- PageTables: 39208 kB
- NFS_Unstable: 0 kB
- Bounce: 0 kB
- WritebackTmp: 0 kB
- CommitLimit: 8172740 kB
- Committed_AS: 14935216 kB
- VmallocTotal: 34359738367 kB
- VmallocUsed: 399340 kB
- VmallocChunk: 34359334908 kB
- HardwareCorrupted: 0 kB
- AnonHugePages: 456704 kB
- HugePages_Total: 0
- HugePages_Free: 0
- HugePages_Rsvd: 0
- HugePages_Surp: 0
- Hugepagesize: 2048 kB
- DirectMap4k: 12288 kB
- DirectMap2M: 16680960 kB
- >>> ipcs -l
- ------ Shared Memory Limits --------
- max number of segments = 4096
- max seg size (kbytes) = 4316816
- max total shared memory (kbytes) = 4316816
- min seg size (bytes) = 1
- ------ Semaphore Limits --------
- max number of arrays = 128
- max semaphores per array = 250
- max semaphores system wide = 32000
- max ops per semop call = 32
- semaphore max value = 32767
- ------ Messages Limits --------
- max queues system wide = 31918
- max size of message (bytes) = 8192
- default max size of queue (bytes) = 16384
sysctl.conf提取,由我计算:
- kernel.shmall = 1079204
- kernel.shmmax = 4420419584
postgresql.conf非默认值,由我计算:
- max_connections = 60 # (change requires restart)
- shared_buffers = 4GB # min 128kB
- work_mem = 4MB # min 64kB
- wal_sync_method = open_sync # the default is the first option
- checkpoint_segments = 16 # in logfile segments,min 1,16MB each
- checkpoint_completion_target = 0.9 # checkpoint target duration,0.0 - 1.0
- effective_cache_size = 6GB
这个合适吗?如果不是(或不一定),在哪种情况下是否合适?
我们确实注意到这个配置有很好的性能改进,你会如何改进它?
如何计算内核内存管理参数?
任何人都可以解释如何从头开始真正设置它们吗?
解决方法
Git fails to push with error ‘out of memory’
我不是在这里回答你的所有问题,而是你标题中的问题:
How to set shmall,shmmax,shmni,etc … in general and for postgresql
对于某些内核发行版,有一些设置会阻止内核将最大内存分配给单个进程:
- Set Kernel Parameters
- Modify the "/etc/sysctl.conf" file to include the lines appropriate to your operating system.
- # Red Hat Enterprise Linux 3.0 and CentOS 3.x
- kernel.shmmax = 2147483648
- kernel.shmmni = 4096
- kernel.shmall = 2097152
- kernel.shmmin = 1
- kernel.shmseg = 10
- # semaphores:
- semmsl,semmns,semopm,semmni kernel.sem = 250 32000 100 128
- fs.file-max = 65536
- # Red Hat Enterprise Linux 4.0 and CentOS 4.x
- kernel.shmmax = 536870912
- kernel.shmmni = 4096
- kernel.shmall = 2097152
如果您的进程超出限制,尽管系统上报告了最大内存,但内核将终止该进程.
注意:请注意这些设置.您可能不希望使用该示例中的设置,因为我从环境中的服务器中提取它们.
一些额外的注意事项:
- To Update and test kernel settings with sysctl,use following commands:
- List current settings: sysctl -A|grep shm
- sysctl -w kernel.shmmax=<value> to write in sysctl.conf
- sysctl -p /etc/sysctl.conf to read/reload the values from sysctl.conf
- Disable secure linux by editing the "/etc/selinux/config" file,making sure the SELINUX flag is set as follows.
- SELINUX=disabled
Is this appropriate ? If not (or not necessarily),in which case would
it be appropriate ?
当ISP提供商不希望单个客户进程占用共享服务器上的所有资源时,通常在数据中心环境中更严格地定义内核设置.
您通常不必设置内核内存参数,除非您有一个由于资源不足而被内核杀死的进程.
在某些情况下,postgres还可以为特定页面大小分配更多内存,而不是共享内存中可用的内容:
- * The Postgresql server Failed to start. Please check the log output:
- 2011-11-04 05:06:26 UTC FATAL: could not create shared memory segment: Invalid
- argument
- 2011-11-04 05:06:26 UTC DETAIL: Failed system call was shmget(key=5432001,size
- =161849344,03600).
- 2011-11-04 05:06:26 UTC HINT: This error usually means that Postgresql’s reques
- t for a shared memory segment exceeded your kernel’s SHMMAX parameter. You can
- either reduce the request size or reconfigure the kernel with larger SHMMAX. To
- reduce the request size (currently 161849344 bytes),reduce Postgresql’s shared
- _buffers parameter (currently 19200) and/or its max_connections parameter (curre
- ntly 53).
- If the request size is already small,it’s possible that it is less than
- your kernel’s SHMMIN parameter,in which case raising the request size or recon
- figuring SHMMIN is called for.
- The Postgresql documentation contains more information about shared memo
- ry configuration.
- …fail!
可以通过调整内核资源设置来解决上述示例等错误.此处详细介绍了用于确定资源设置的建议设置和方法:
http://www.postgresql.org/docs/9.1/static/kernel-resources.html
但是,除非遇到与postgres进程相关的资源不足情况,否则您实际上不必触摸这些设置.这些情况通常发生在分配给它们的资源很少的共享环境或服务器中.
Can anybody explain how to really set them from the ground up ?
对于Postgres调整,您应该阅读:
http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server