Skip to content

Word2vec on GPU slower than CPU #13048

@manneshiva

Description

@manneshiva

System information

  • OS Platform and Distribution: Linux Ubuntu 16.04
  • TensorFlow installed from (source or binary): source
  • TensorFlow version (use command below): 1.3.0
  • Python version: 2.7.12
  • Bazel version (if compiling from source): 0.5.0
  • CUDA/cuDNN version: 8.0/6.0
  • GPU model and memory: NVIDIA GTX 1060 / 3GB
  • Docker used: yes
  • I picked up the code from the word2vec example on your official repo and made a few changes.The core code to train word2vec remains the same.

Describe the problem

I have been working on benchmarking commonly used frameworks/libraries for unsupervised learning of word embeddings(word2vec). I am currently comparing tensorflow(cpu/gpu), gensim, deeplearning4j and the original c code on standard metrics like training time, peak memory usage and quality of learned vectors. Link to my github repo (still working on it). I ran the benchmark on text8 corpus(plan to run it on a much larger corpus later for the true picture) which gave me strange results.

  • Tensorflow on GPU is much slower than CPU
  • Tensorflow is much slower than other frameworks

Is this behavior expected? Would appreciate any inputs.

Source code / logs

Link to tensorflow code
Link to results of sample benchmark on text8 corpus

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions