Use zram to reduce the memory of program

zram is a driver in linux kernel. It compress the content in memory to reduce the pages used by application.

But that’s not the only way we could use zram. Furthermore, we could use zram with tcmalloc to reduce user application’s cost of memory.

Now we make redis-server to use memory in zram. If we use “lsof” to check the redis-server, it will show:

That’s the file created by tcmalloc library, and it has already be compressed.

C++/Java developers needed

I worked in Alibaba Group for more than 9 years. Recently I am working in Alimama, a sub-company of Alibaba Group and has been the biggest Advertisement Publishing Company in China. At present, we need C++/Java developers to build new back-end basic services for our new business.

[Job Description]

Role: C++/Java Developer for storage system or high performance computing

Location: Beijing

Your responsibilities:

1. Building and optimizing the distributed key-value storage system
2. Building and optimizing the distributed computing engine of Linear Regression algorithm
3. Building and maintaining the backend service for Advertisement Publishing System

Skins & experience required:

1. Familiar with storage system or hight performance computing system
2. Strong background about Redis/Rocksdb/Hadoop/Glusterfs
3. Very familiar with one of C/C++/Java/Scala language
4. More than 3 years experience about storage system or HPC as a developer
5. Passionate about new Technologies and wanting to continuously push the boundaries

Any one who is interesting in the job above could send email to my email: haodong@alibaba-inc.com

Some problems and solutions when deploying and running Hadoop-2.7.2

1. If we see this error report:

The solution is here, the heap size of Java should not be bigger than map/reduce memory. The Cloudera recommends the head size prefer to be 0.8 of the map/reduce memory, such as:

2. The directory of “/tmp/” became full.

This is usually caused by spilled data from map output. This article introduced the whole overview of Map/Reduce algorithm in Hadoop with a detailed and clear picture.
As a result, my solution is adding this configuration:

into core-site.xml, so the inevitable spill data will be write into different disks for load balance.

3. Don’t use more than 0.8 of physical memory as “yarn.nodemanager.resource.memory-mb”, or it will cause unexpected fail for jobs.

4. If we launch too many map jobs or reduce jobs more than physical cores of servers, it may lead to tremendous timeouts for these jobs. Therefore, adjust the “mapreduce.map.memory.mb” and “mapreduce.reduce.memory.mb” carefully to limit the number of map/reduce jobs.

5. If you notice that all the CPU cores are full in Hadoop cluster, that does not mean we can’t do optimizations anymore. By using perf, I find out system waste too many times on launching and stopping java task (or containers):

hadoop

So I change the value of “mapreduce.input.fileinputformat.split.minsize” to 8GB for reducing the number of mappers. After decrease the number of mappers from thousands to hundreds, the running time of Terasort program drop down more than 50% (Also the Context Switch of system fall from ten thousands per second to thousands). Therefore, adjust the number of java tasks close to the number of physical CPU cores is a better solution.