When using network attached storage, Linux and other OSes are vulnerable to a fundamental deadlock problem. The premise of the problem is that one may need memory in order to free memory, and freeing memory is done mostly when there is very little free memory left.

The problem can happen with normally written files, MAP_SHARED mmaped files and swap, but is most likely to happen with the latter two. It can hit with NFS, iSCSI, AoE or any other network storage protocol, since they all rely on the network layer.

The bug can get triggered as follows:

• the system is low on free memory
  • as a result, kswapd starts trying to free up memory by evicting pages
  • if the pages are dirty, they have to be written back to storage
• writing pages back to storage over network requires allocating memory
  • memory for packet headers (if the NIC can assemble the packet itself)
  • memory for entire packets
• kswapd can get this memory, because it has higher priority and the system sets aside something extra for the pageout code

• now the system is really low on memory, it may even have no free memory left at all

• the NAS appliance receives the write request from the computer
  • the NAS appliance sends back an ACK packet acknowledging that the data was received
  • the NAS appliance sends back a packet acknowledging the OS that the data was written to disk

• however, at this point the OS may not have any memory left to receive these packets from the NAS

• the OS never knows whether the I/O has completed, since it cannot receive any more network packets
• the computer deadlocks