Maybe it was my CS major talking there. An algorithm is a sequence of steps to reach a desired outcome such as updating a neural network. The network itself is essentially just a big heap of values you multiply through if you were curious.