InvalidArgumentError (см. Выше для отслеживания): не было зарегистрировано OpKernel для поддержки Op 'MklConv2DWithBias' с этими attr

Я клонировал tensorflow 1.1.0, изменил файл ./configure, включил опцию mkl:

## Set up MKL related environment settings
if true; then # modify this to be true.
  while [ "$TF_NEED_MKL" == "" ]; do
    fromuser=""
    read -p "Do you wish to build TensorFlow with MKL support? [y/N] " INPUT
    fromuser="1"
    case $INPUT in
      [Yy]* ) echo "MKL support will be enabled for TensorFlow"; TF_NEED_MKL=1;;
      [Nn]* ) echo "No MKL support will be enabled for TensorFlow"; TF_NEED_MKL=0;;
      "" ) echo "No MKL support will be enabled for TensorFlow"; TF_NEED_MKL=0;;
      * ) echo "Invalid selection: " $INPUT;;
    esac
  done

затем используйте команду. /configure:

$ ./configure
Please specify the location of python. [Default is /usr/local/bin/python]:
Do you wish to build TensorFlow with MKL support? [y/N]
No MKL support will be enabled for TensorFlow
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: ^C
root@4be8a8788f34:/tensor/tensorflow# ./configure
Please specify the location of python. [Default is /usr/local/bin/python]:
Do you wish to build TensorFlow with MKL support? [y/N] y
MKL support will be enabled for TensorFlow

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Found possible Python library paths:
  /usr/local/lib/python2.7/site-packages
  /tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy
Please input the desired Python library path to use.  Default is [/usr/local/lib/python2.7/site-packages]

Using python library path: /usr/local/lib/python2.7/site-packages
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N]
No CUDA support will be enabled for TensorFlow
Configuration finished
......................
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
.................
INFO: All external dependencies fetched successfully.

Я скомпилировал shadoworflow со следующей командой:

bazel build --config=opt --config=mkl --verbose_failures //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

Затем установите его с помощью следующей команды:

pip install --target=$(pwd) /tmp/tensorflow_pkg/tensorflow-1.1.0-cp27-cp27mu-linux_x86_64.whl

Я использовал этот код для тестирования mkl, включенный в тензорном потоке, упомянутый в комментариях tensorflow-build-cpu-mkl-windows:

from tensorflow.python.ops import nn_ops
import tensorflow as tf
import numpy as np

images = np.ones((1,1,15,1)).astype(np.float32)
filters = 1 * np.ones((1,1,1,1), np.float32)

with tf.Session(''):
    output = nn_ops.conv2d(
        images,
        filters,
        strides=[1,1,1,1],
        padding='VALID',
        data_format='NCHW',
    ).eval()
    print output

Код работает успешно:

[[[[ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]
   [ 1.]]]]

Из кода вы можете видеть, что использование nn_ops.conv2d не представляет проблемы. Но проблема возникает при использовании slim.conv2d:

# demo from https://www.jianshu.com/p/a70c1d931395
import tensorflow as tf 
import tensorflow.contrib.slim as slim

x1 = tf.ones(shape=[1, 64, 64, 3]) 
w = tf.fill([5, 5, 3, 64], 1)
# print("rank is", tf.rank(x1))

# x1 = tf.cast(x1, tf.float32)
w = tf.cast(w, tf.float32)

print('-----debugging-----')
print(type(x1))
print(x1.dtype.base_dtype)

print(type(w))
print(w.dtype.base_dtype)
print('-------------------')

# x1 = tf.cast(x1, tf.float16)

y1 = tf.nn.conv2d(x1, w, strides=[1, 1, 1, 1], padding='SAME')
y2 = slim.conv2d(x1, 64, [5, 5], weights_initializer=tf.ones_initializer, padding='SAME')

with tf.Session() as sess: 
    sess.run(tf.global_variables_initializer()) 
    y1_value,y2_value,x1_value=sess.run([y1,y2,x1])
    print("shapes are", y1_value.shape, y2_value.shape)
    print(y1_value==y2_value)
    print(y1_value)
    print(y2_value)

Возникла проблема:

-----debugging-----
<class 'tensorflow.python.framework.ops.Tensor'>
<dtype: 'float32'>
<class 'tensorflow.python.framework.ops.Tensor'>
<dtype: 'float32'>
-------------------

Traceback (most recent call last):
  File "/code/test_conv2d.py", line 26, in <module>
    sess.run(tf.global_variables_initializer())
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/client/session.py", line 778, in run
    run_metadata_ptr)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/client/session.py", line 982, in _run
    feed_dict_string, options, run_metadata)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/client/session.py", line 1032, in _do_run
    target_list, options, run_metadata)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/client/session.py", line 1052, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'MklConv2DWithBias' with these attrs.  Registered devices: [CPU], Registered kernels:
  device='CPU'; label='MklLayer'; T in [DT_FLOAT]

     [[Node: Conv/BiasAdd = MklConv2DWithBias[T=DT_FLOAT, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](ones, DMT/_0, Conv/weights/read, DMT/_1, Conv/biases/read, DMT/_4)]]

Caused by op u'Conv/BiasAdd', defined at:
  File "/code/test_conv2d.py", line 23, in <module>
    y2 = slim.conv2d(x1, 64, [5, 5], weights_initializer=tf.ones_initializer, padding='SAME')
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/contrib/framework/python/ops/arg_scope.py", line 182, in func_with_args
    return func(*args, **current_args)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/contrib/layers/python/layers/layers.py", line 924, in convolution
    outputs = layer.apply(inputs)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/layers/base.py", line 323, in apply
    return self.__call__(inputs, **kwargs)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/layers/base.py", line 292, in __call__
    outputs = self.call(inputs, **kwargs)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/layers/convolutional.py", line 176, in call
    data_format=utils.convert_data_format(self.data_format, 4))
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/ops/nn_ops.py", line 1346, in bias_add
    return gen_nn_ops._bias_add(value, bias, data_format=data_format, name=name)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/ops/gen_nn_ops.py", line 282, in _bias_add
    data_format=data_format, name=name)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/framework/op_def_library.py", line 779, in apply_op
    op_def=op_def)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/framework/ops.py", line 2336, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/tensor/target/applib-runtime-tensorflow-mkl-with-cpu-without-numpy/tensorflow/python/framework/ops.py", line 1228, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): No OpKernel was registered to support Op 'MklConv2DWithBias' with these attrs.  Registered devices: [CPU], Registered kernels:
  device='CPU'; label='MklLayer'; T in [DT_FLOAT]

     [[Node: Conv/BiasAdd = MklConv2DWithBias[T=DT_FLOAT, data_format="NHWC", padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](ones, DMT/_0, Conv/weights/read, DMT/_1, Conv/biases/read, DMT/_4)]]

Если я не использую mkl оптимизированный тензорный поток, такой проблемы нет.

Я вижу, что MklConv2DWithBias зарегистрирован в tensorflow/core/ops/nn_ops.cc:

REGISTER_OP("MklConv2DWithBias")
    .Input("input: T")
    .Input("mkl_input: uint8")
    .Input("filter: T")
    .Input("mkl_filter: uint8")
    .Input("bias: T")
    .Input("mkl_bias: uint8")
    .Output("output: T")
    .Output("mkl_output: uint8")
    .Attr("T: {half, float, double}")
    .Attr("strides: list(int)")
    .Attr("use_cudnn_on_gpu: bool = true")
    .Attr(GetPaddingAttrString())
    .Attr(GetConvnetDataFormatAttrString());

Я вижу здесь no-opkernel-is-registered-to-support-op-conv2d-with-these-attrs говорят, что вход и фильтр должны использовать float32, я уверен, что input - float32, но как установить фильтр для float32 здесь?

Я попытался преобразовать тип параметров, но не смог. Ошибка все еще существует::

slim.conv2d(x1, np.float32(64.0), np.array([5.0, 5.0], dtype=np.float32), weights_initializer=tf.ones_initializer, padding='SAME')

Как создать проблему

Я нашел способ воспроизвести эту проблему.

1.загрузить тензорный поток zip и распаковать.

https://drive.google.com/file/d/1k616LEvgTUXpHuX6Fz7EDXukaG5tGZwQ/view

2.mount папка tensorflow в контейнер:

docker run --rm -it -v $(pwd):/code econtal/numpy-mkl  bash

3. создать тестовый скрипт:

$ cd /code
$ vi test_conv2d.py

# demo from https://www.jianshu.com/p/a70c1d931395
import tensorflow as tf 
import tensorflow.contrib.slim as slim
import numpy as np

x1 = tf.ones(shape=[1, 64, 64, 3]) 
w = tf.fill([5, 5, 3, 64], 1)
# print("rank is", tf.rank(x1))

# x1 = tf.cast(x1, tf.float32)  
w = tf.cast(w, tf.float32)

print('-----debugging-----')
print(type(x1))
print(x1.dtype.base_dtype)

print(type(w))
print(w.dtype.base_dtype)
print('-------------------')

# x1 = tf.cast(x1, tf.float16)

y1 = tf.nn.conv2d(x1, w, strides=[1, 1, 1, 1], padding='SAME')
y2 = slim.conv2d(x1, np.float32(64.0), np.array([5.0, 5.0], dtype=np.float32), 
weights_initializer=tf.ones_initializer, padding='SAME')

with tf.Session() as sess: 
    sess.run(tf.global_variables_initializer()) 
    y1_value,y2_value,x1_value=sess.run([y1,y2,x1])
    print("shapes are", y1_value.shape, y2_value.shape)
    print(y1_value==y2_value)
    print(y1_value)
    print(y2_value)

4.run тестовый скрипт $ python test_conv2d.py

Может ли кто-нибудь дать совет о том, как найти проблему? Благодарю.