Difference between np.dot and np.multiply with np.sum in binary cross-entropy loss calculation

Asad Shakeel picture Asad Shakeel · Jan 11, 2018 · Viewed 38.7k times · Source

I have tried the following code but didn't find the difference between np.dot and np.multiply with np.sum

Here is np.dot code

logprobs = np.dot(Y, (np.log(A2)).T) + np.dot((1.0-Y),(np.log(1 - A2)).T)
print(logprobs.shape)
print(logprobs)
cost = (-1/m) * logprobs
print(cost.shape)
print(type(cost))
print(cost)

Its output is

(1, 1)
[[-2.07917628]]
(1, 1)
<class 'numpy.ndarray'>
[[ 0.693058761039 ]]

Here is the code for np.multiply with np.sum

logprobs = np.sum(np.multiply(np.log(A2), Y) + np.multiply((1 - Y), np.log(1 - A2)))
print(logprobs.shape)         
print(logprobs)
cost = - logprobs / m
print(cost.shape)
print(type(cost))
print(cost)

Its output is

()
-2.07917628312
()
<class 'numpy.float64'>
0.693058761039

I'm unable to understand the type and shape difference whereas the result value is same in both cases

Even in the case of squeezing former code cost value become same as later but type remains same

cost = np.squeeze(cost)
print(type(cost))
print(cost)

output is

<class 'numpy.ndarray'>
0.6930587610394646

Answer

Anuj Gautam picture Anuj Gautam · Jan 11, 2018

np.dot is the dot product of two matrices.

|A B| . |E F| = |A*E+B*G A*F+B*H|
|C D|   |G H|   |C*E+D*G C*F+D*H|

Whereas np.multiply does an element-wise multiplication of two matrices.

|A B| ⊙ |E F| = |A*E B*F|
|C D|   |G H|   |C*G D*H|

When used with np.sum, the result being equal is merely a coincidence.

>>> np.dot([[1,2], [3,4]], [[1,2], [2,3]])
array([[ 5,  8],
       [11, 18]])
>>> np.multiply([[1,2], [3,4]], [[1,2], [2,3]])
array([[ 1,  4],
       [ 6, 12]])

>>> np.sum(np.dot([[1,2], [3,4]], [[1,2], [2,3]]))
42
>>> np.sum(np.multiply([[1,2], [3,4]], [[1,2], [2,3]]))
23