The role of bias in Neural Networks

The activation function in Neural Networks takes an input ‘x’ multiplied by a weight ‘w’. Bias allows you to shift the activation function by adding a constant (i.e. the given bias) to the input. Bias in Neural Networks can be thought of as analogous to the role of a constant in a linear function, whereby the line is effectively transposed by the constant value.

In a scenario with no bias, the input to the activation function is ‘x’ multiplied by the connection weight ‘w₀‘.

In a scenario with bias, the input to the activation function is ‘x’ times the connection weight ‘w₀‘ plus the bias times the connection weight for the bias ‘w₁‘. This has the effect of shifting the activation function by a constant amount (b * w₁).

Nature of Bias and its implications
Bias could be at individual, organisational or decision making levels, many at times bias could be positive or a heuristic process leading to advantage.  However in many occasions organisation could get caught in the vicious circle of bias eventually affecting business performance..

How do we have a bias free organisations.
The easiest option is not to have “your favourite” going on in your organisation.  During my interactions with organisations in middle markets and family driven businesses, the process and system designs are based either on legacy or “my way” policy…  it may be yielding myopic results, but here is where bias starts to slither in the system…

In my view and thru the lens of behavioural science, businesses should focus on processes, not people. Behavioural science tells us that it’s very hard to eliminate our biases, but we can redesign organizations to circumvent them.  Behavioural approach as a design or metrics would allow organisations to have checks and balances and it will be easier to do the unbiased thing either by preventing biased choices or altering heuristic types.