Monthly Archives: September 2016

How to show the memory details of a process in Linux

Recently we are evaluating the memory consumption of redis process when it is rewriting of append-only-file. In this situation, redis will firstly fork(), and then the child process will write all the data from memory to new AOF file and the parent process will still provide service to custom. The… Read more »

“java.io.Exception: failed to uncompress the chunk” in Apache Spark

After I run spark-submit in my YARN cluster with Spark-1.6.2:

The job fail, and the log report:

Somebody in the internet say may be this is caused by the compatibility problem between Spark-1.6.2 and Snappy. Therefore I add

to my spark-submit shell script to change compress algorithm… Read more »

Finding the lost memory

      No Comments on Finding the lost memory

We find out a strange phenomenon in a product server. By using “free” command, it shows there is no free memory in this server. But when we add all processes’s memory allocation:

it show all processes cost only 60GB memory (The whole physical memory of this server is 126GB)…. Read more »