Author Archives: Robin Dong

Compare implementation of tf.AdamOptimizer to its paper

When I reviewed the implementation of Adam optimizer in tensorflow yesterday, I noticed that it’s code is different from the formulas that I saw in Adam’s paper. In tensorflow’s formulas for Adam are: But the algorithm in the paper is: Then quickly I found these words in the document of… Read more »

The bug about using hooks and MirroredStrategy in tf.estimator.Estimator

When I was using MirroedStrategy in my tf.estimator.Estimator:

and add hooks for training:

The tensorflow report errors:

Without finding any answers on google, I have to look into the code of ‘estimator.py’ in tensorflow. Fortunately, the code defect is obvious:

class Estimator havn’t any private argument… Read more »

Some lessons from Kaggle’s competition

About two months ago, I joined the competition of ‘RSNA Pneumonia Detection’ in Kaggle. It’s ended yesterday, but I still have many experiences and lessons to be rethinking. 1. Augmentation is extremely crucial. After using tf.image.sample_distorted_bounding_box() in my program, the mAP(mean Average Precision) of evaluating dataset thrived to a perfect… Read more »

How could it possible to assign an integer to string?

The snippet below could be compiled and run:

The result is:

I noticed that the corresponding value of key ‘banana’ is empty. The reason is I assign an integer directly to key ‘banana’ by mistake. But how could c++ compiler allow me to do this? Why doesn’t it… Read more »

Move semantics in C++11

      No Comments on Move semantics in C++11

After studying an example for Move Semantics of C++11, I write a more complete code snippet:

Pay attention to last two lines in ‘move constructor’:

Since ‘move constructor’ will not set initial value for m_size and m_data of ‘v4’, the m_size and m_data of ‘v3’ will be uninitial… Read more »

How Tensorflow set device for each Operation ?

In Tensorflow, we only need to use snippet below to assign a device to a Operation:

How dose it implement? Let’s take a look. There is a mechanism called ‘context manager’ in Python. For example, we can use it to add a wrapper for a few codes:

The… Read more »

Some tips about using google’s TPU (Cont.)

Sometimes I get this error from TPUEstimator:

And after stop and restart TPU in console of GCP, the error disappeared. TPU doesn’t allow users to use it directly like GPU. You can’t see the device in VM looks like ‘/dev/tpu’ or something like this. Google provides TPU as RPC… Read more »

Some modifications about SSD-Tensorflow

In the previous article, I introduced a new library for Object Detection. But yesterday, after I added slim.batch_norm() into ‘nets/ssd_vgg_512.py’ like this:

Although training could still run correctly, the evaluation reported errors:

I wondered why adding some simple batch_norm will make shape incorrect for quite a while. Finally… Read more »

Choosing a Object Detection Framework written by Tensorflow

Recently I need to train a DNN model for object detection task. In the past, I am using the object detection framework from tensorflows’s subject — models. But there are two reasons that I couldn’t use it to do my own project: First, it’s too big. There are more than… Read more »