Thu 09 Jan 2020

Copying data is wasteful, mutating data is dangerous

You have a large chunk of data—a NumPy array, or a Pandas DataFrame—and you need to do a series of operations on it. By default both libraries make copies of the data, which means you’re using even more RAM.

Both libraries do have APIs for modifying data in-place, but that can lead to other problems, including subtle bugs.

So what can you do?

In this article you’ll learn to recognize and apply the “interior mutability” pattern, which offers a compromise between the two: the safe operation of copy-based APIs, with a somewhat reduced memory usage.

Source: Copying data is wasteful, mutating data is dangerous, an article by Itamar Turner-Trauring.