0

Wikipedia has the following example code for softmax.

>>> import numpy as np
>>> z = [1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0]
>>> softmax = lambda x : np.exp(x)/np.sum(np.exp(x))
>>> softmax(z)
array([0.02364054, 0.06426166, 0.1746813 , 0.474833  , 0.02364054 , 0.06426166, 0.1746813 ])

When I run it, it runs successfully. I don't understand how to read the lambda function. In particular, how can the parameter x refer to an array element in the numerator and span all the elements in the denominator?

[Note: The question this question presumably duplicates is about lambdas in general. This question is not necessarily about lambda. It is about how to read the np conventions. The answers by @Paul Panzer and @Mihai Alexandru-Ionut both answer my question. Too bad I can't check both simultaneously as answering the question.

To confirm that I understand their answers (and to clarify what my question was about):

  • x is the entire array (as it should be since the array is passed as the argument).
  • np.exp(x) returns the array with each element x[i] replaced by np.exp(x[i]). Call that new array x_new.
  • x_new/np.sum(x_new) divides each element of x_new by the sum of x_new.

]

RussAbbott
  • 2,660
  • 4
  • 24
  • 37

3 Answers3

1

A lambda expression is like an anonymous function. In this context, the line

softmax = lambda x : np.exp(x)/np.sum(np.exp(x))

is equivalent to

def softmax(x):
    return np.exp(x)/np.sum(np.exp(x))
Antwane
  • 20,760
  • 7
  • 51
  • 84
1

I don't understand how to read the lambda function.

A lambda function is an anonymous function that is not bound to a name

In particular, how can the parameter x refer to an array element in the numerator and span all the elements in the denominator?

No, parameter x does not refer to an array element, but to the whole array. np.exp(x) method also returns a new array.

By writing np.exp(x)/np.sum(np.exp(x)) the array resulted from np.exp(x) will be divided to the sum and it means that every item from the array will be divided to that sum.

Mihai Alexandru-Ionut
  • 47,092
  • 13
  • 101
  • 128
1

Three remarks.

The use of lambda in the example is actually bad style, cf. this paragraph from the Python style guide:

Always use a def statement instead of an assignment statement that binds a lambda expression directly to an identifier.

Yes:

def f(x): return 2*x

No:

f = lambda x: 2*x

The first form means that the name of the resulting function object is specifically 'f' instead of the generic ''. This is more useful for tracebacks and string representations in general. The use of the assignment statement eliminates the sole benefit a lambda expression can offer over an explicit def statement (i.e. that it can be embedded inside a larger expression)

Re the content. What you are seeing is array arithmetic. np.exp is a numpy ufunc it operates element-wise, so it will return an array of the same shape as its argument. np.sum is a reducing function, when called with an array as its sole argument it will return a scalar. The / operator is overloaded with a binary ufunc; like np.exp it operates element-wise. In addition, it does broadcasting: In this case the scalar denominator will be paired with every element of the array numerator resulting in an array.

And finally: Here is how to implement the softmax properly.

Paul Panzer
  • 51,835
  • 3
  • 54
  • 99
  • When I use `def` to define the funciton: `def fsoftmax(x): np.exp(x)/np.sum(np.exp(x))` and then `print(fsoftmax(z))` I get `None` instead of the vector. Why is that? – RussAbbott May 22 '18 at 16:17
  • @RussAbbott, you have to use `return` statement : `def fsoftmax(x): return np.exp(x)/np.sum(np.exp(x))` – Mihai Alexandru-Ionut May 22 '18 at 16:21
  • Because a standard function have to return a result, using a return statement `return np.exp(x)/np.sum(np.exp(x))`. A lambda doesn't need a return statement. See https://stackoverflow.com/questions/37024343/why-cant-i-use-return-in-lambda-function-in-python – Antwane May 22 '18 at 16:21
  • OK. Makes sense. – RussAbbott May 22 '18 at 21:59