Analyse and visualise Linux OOM output

OOMAnalyser is a small project to transform the OOM message of a Linux kernel into a more user-friendly format.

OOMAnalyser consists of a web page into whose input field the OOM message is copied. JavaScript code extracts the data from it and displays the details. All processing takes place in the browser. No data is transferred to external servers. This makes it possible to use a locally stored copy of the website for analysis. This project is written in Python and uses Transcrypt to translate Python code into JavaScript.

Step 1 - Enter your OOM message


Step 2 - Results

Go back to "Step 1 - Enter your OOM message" to run a new analysis.

Summary

The process "" (pid ) requested () memory. The system couldn't satisfy this request and started the OOM killer to free memory. The OOM killer calculates a score for each process and terminates the process with the highest score. It terminates "" (pid ) to satisfy the initial memory request.

The terminated process uses () resident memory and has an OOM score of .

The system has ( * ) physical memory and swap space. That's total.

Details of analysis

The result of the analysis is displayed in three columns. The first column is used to name the property including the original OOM identifier in brackets. The extracted information is displayed in the second column. The last column contains further details and additional information.

Tigger Process
(PID ) This process requests memory and is triggering thereby the OOM situation.
Memory allocation flags
(gfp_mask)
These flags are used to control the kernel internal memory allocation
GFP stands for __get_free_pages().
Node mask to show on which CPU Cores this process can run
(nodemask)
Bit mask indicating the cores on which the process can run.
Requested memory
(order)
(2) pages / The kernel specifies the requested number of pages as exponent of power of two.
Adjust oom-killer score
(oom_score_adj)
This value is added to the badness score before it's used to determine the process to be killed.
Killed Process
(PID ) Process killed by Linux kernel to satisfy the memory request.
OOM Score
(score)
Programs with the highest OOM score are terminated first.
Virtual Memory
(total_vm)
Virtual memory used by this process.
Resident anonymous memory
(rss)
Part of the virtual process memory mapped into RAM.
Memory Usage Graphs
RAM Summary
Swap Summary
Memory Pages
RAM pages Total number of RAM pages
HighMem/MovableOnly Number of pages in the High Memory Area or marked movable for Contiguous Memory Allocator (CMA).
HighMem pages are also counted in the total page number.
Reserved pages Number of reserved pages
CMA reserved pages 0 Pages reserved for Contiguous Memory Allocator (CMA)
Pagetable Cache 0 Number of pages in pagetable cache
Number of pages with hardware errors 0 Pages with uncorrectable memory errors
Memory Usage Details
Active anonymous memory
(active_anon)
Recently used anonymous memory.
These memory pages will usually not swapped out.
Inactive anonymous memory
(inactive_anon)
Least recently used anonymous memory.
These memory pages can be swapped out.
Isolated anonymous memory
(isolated_anon)
Memory isolation is used to separate memory between different virtual machines.
Active Pagecache
(active_file)
Pagecache that has been used more recently and usually not reclaimed unless absolutely necessary.
Inactive Pagecache
(inactive_file)
Pagecache which has been less recently used. It can be reclaimed without huge performance impact.
Isolated Pagecache
(isolated_file)
Memory isolation is used to separate memory between different virtual machines.
Unevictable Pages
(unevictable)
Unevictable memory. It can't be swapped out because the pages are owned by ramfs or protected by mlock(3) / shmctl(SHM_LOCK). Unevictable pages are managed by kernels LRU framework.
Dirty Pages
(dirty)
Memory which is waiting to get written back to the disk. [1]
Writeback
(writeback)
Memory which is actively being written back to the disk. [1]
Unstable
(unstable)
Not yet committed to stable storage.
Slab Reclaimable
(slab_reclaimable)
Slab is a in-kernel data structures cache. Part of Slab, that might be reclaimed, such as caches. [1]
Additional details are listed in slabinfo(5) also.
Slab Unreclaimable
(slab_unreclaimable)
Part of Slab, that cannot be reclaimed on memory pressure. [1]
Mapped
(mapped)
Files which have been mapped into memory (with mmap(2)), such as libraries. [1]
Shared Memory
(shmem)
Amount of memory consumed in tmpfs(5) filesystems. [1]
Pagetables
(pagetables)
Amount of memory dedicated to the lowest level of pagetables. [1]
Bounce
(bounce)
Memory used for block device "bounce buffers". [1]
free
(free)
free_pcp
(free_pcp)
free_cma
(free_cma)
Total Pagecache
Swap Usage
Swap Total Total amount of swap space available. [1]
Swap Free Amount of swap space that is currently unused. [1]
Swap Cached Memory that once was swapped out, is swapped back in but still also is in the swap file. (If memory pressure is high, these pages don't need to be swapped out again because they are already in the swap file. This saves I/O). [1]
Swap Used Amount of used swap space w/o cached swap
(SwapUsed = SwapTotal - SwapFree -SwapCache)
Operating System
Kernel
Distribution Guessed from the kernel version
Platform Guessed from the kernel version
Page size Guessed
Memory Chunks

            
Process Table

            
Hardware Details

            
Kernel Call Trace

            
Entire OOM Message (click to hide)

            

Go back to "Step 1 - Enter your OOM message" to run a new analysis.

Footnotes

  1. proc(5) - process information pseudo-filesystem (Go Back)

Further Information

  1. Linux man pages online
  2. Decoding the Linux kernel's page allocation failure messages
  3. Linux Kernel OOM Log Analysis

Changelog

(click to show / hide)

Version 0.4.0 - 2020-XX-XX:

General

  1. Add a textual summary of the analysis
  2. Fix interpretation of the last OOM line
  3. ...

Note

See the commit history of the repository for a full list of changes.

Version 0.3.0 - 2019-11-24:

General

  1. Improve presentation
  2. Separate analysis and visualisation code
  3. Use CSS classes to control the visibility
  4. Strip columns left to the message automatically
  5. Lot if internal improvements and restructuring
  6. Bug fixes

Note

See the commit history of the repository for a full list of changes.

Local Installation

(click to show / hide)
Installing OOMAnalyser is quite easy since OOMAnalyser consists only of two files, a HTML file and a JavaScript file. Both can be stored locally to use OOMAnalyser without an Internet connection.

Installation steps

  1. Create an own directory and add a __javascript__ subdirectory
  2. Download the HTML file to the main directory and the JavaScript file to the __javascript__ subdirectory
  3. Open the file OOMAnalyser.html in your favourite browser.

Copyright (c) 2017-2020 Carsten Grohmann mail <add at here> carsten-grohmann.de

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.