Essential tips to Optimize Your Python Code

It’s always a good idea to get into the habit of coding python efficiently and spotting the places where you can improve immediately.

An essential thing to remember when you are looking for approaches to upgrade your code is that there will most probably always be some trade-offs to accept. For instance, it’s either a faster running bit of code or a simpler one. And simplicity here doesn’t mean just “code that looks less cool” and also, it’s going to be easy to maintain and test.

Additionally, every tip on best way to optimize your Python code needs to be critically inspected with regards to your case. There are general observations but you should also have a context in which those observations are unimportant or even yield the contrary result. So, you need to comprehend what’s going on “behind the scenes” and how it’ll work in your situation.

However, there are certain situations where you need to boost your code a little bit and here are some of the points that will be helpful when learning Python.

1. List comprehensions

In Python, an extraordinary syntactic constructor that is computationally more efficient than traditional loop for creating lists list comprehensions. So, in the event that you need to, say, have a binary feature vector for a list of numbers as data points where all negative numbers will be assigned 0 and the rest will be assigned 1, rather than:

>>> input_list = [1, 2, -3]
>>> output_list = []

>>> for x in input_list:

...    if x >= 0:

...    output_list.append(1)

...    else:

...        output_list.append(0)

>>> output_list

[1, 1, 0]

You can also do:

>>> output_list = [1 if x >= 0 else 0 for x in input_list]

>>> output_list

[1, 1, 0]

You can just try and compare which implementation runs faster using, for instance, the timeit module.

You can also implement nested list comprehensions similar to nested loops but due to its complexity in reading, maintaining and test it is generally discouraged.

2. Avoid for-loops and list comprehensions where possible

In fact, in the previous example, if you were creating a vector with just one initialization value, rather than using inefficient for-loops and even list comprehensions, you could accomplish something like that:

>>> my_list2 = [0] * len(my_list1)

>>> my_list2

[0, 0, 0]

3. Avoid unnecessary functions

A good approach that can assist shave off a lot of the runtime complexity however requires a lot of careful idea in terms of trade-offs is function calls. If you want the nice abstraction, extensibility, and re-usability that functions offer, it is not required to have a function for every single thing, because function calls are very expensive in Python. So,at times, you might want to sacrifice, for instance, writing a getter and/or a setter. Then again, fewer functions make the code less testable. Hence, the final decision completely depends on the points of interest of your application.

4. Use built-ins where possible

Another tip related to functions is going for built-in functions like max(), sum(), map(), reduce(), etc. instead of carrying out those calculations yourself — they are normally written in C and will run quicker. Additionally, if you utilize a built-in function — it’s less code you will have to write tests for yourself. So, for instance, if you need a set consisting of all absolute values of input_list, you could implement something like below:

>>> output_set = set(map(abs, input_list))

>>> output_set

{1, 2, 3}

While working with textual data, for string concatenation, instead of using +=:

>>> sentence_list = ['This ', 'is ', 'i2tutorials.']
>>> sentence = ''
>>> for i in sentence_list:
...     sentence += i
>>> sentence
'This is i2tutorials.'

use str.join():

>>> sentence = ''.join(sentence_list)
>>> sentence
'This is i2tutorials.'

Python allocates memory for each intermediate string when using +=, and only once if str.join()  is used.

Implement operator.itemgetter for sorting. If you have a list of tuples, for example, first and last names like this:

>>> my_tuples =[('abbie', 'smith'), ('jane', 'adams'), ('adam', 'johnson')]

Default sorting would have returned this:

>>> sorted(my_tuples)
[('abbie', 'smith'), ('adam', 'johnson'), ('jane', 'adams')]

If you want it to be sorted by last names instead of first names, you can do it like so:

>> sorted(my_tuples, key=operator.itemgetter(1))
[('jane', 'adams'), ('adam', 'johnson'), ('abbie', 'smith')]

5. Avoid the dot

If you are using some of its properties of an object that you have, assign them to local variables first:

rectangle_height = rectangle.height

rectangle_width = rectangle.width

So, iflater your using in code you’re computing, for instance, its surface, you will write as below:

surface = rectangle_height * rectangle_width

And later on if you also wants to compute its perimeter, you will re-use the same variables:

perimeter = 2 * rectangle_height + 2 * rectangle_width

You can utilize the timeit module again to confirm that it saves you the query time for each reference to the rectangle object and its properties.

For similar reasons, utilizing globals is frequently discouraged — you would prefer not to waste time looking up, first, the global variable itself, and afterward each of its properties you may be referring to.

This can likewise apply to function references. In case you’re processing a sequence of data points and you’re appending each subsequent result item to a list, you can perform the following:

push_item = my_list.append

and then apply that to each resulting item:

push_item(item)

6. Know your data structures and know how they work in your version of Python

In addition to general properties of each typical data structure the complexity of recovering an item from a linked list, it’s good to know how Python data structures are actualized and where you can save some CPU or memory there. For instance, if you’re looking into a key in a dictionary, you don’t even have to reference dict.keys() which makes it a little slower in Python 3. You can simply just do:

>>> if k in dict:
...    do_something(k)

In Python 2, dict.keys() is also used to create an extra list of the keys!

7. Choose an approach wisely

Simultaneously, remember to take a look at the bigger picture. If you had a list of things to process and you know that a lookup in a set is O(1) vs O(n) in a list, you may be curious to turn your list into a set:

>>> my_set = set(my_list)

In any case, if you’re only looking up one item of that list, you may really make things worse that you turn your list into a set first. In the background, Python will iterate over your whole list and add every item to a newly-created set. The overhead of creating a set will make you lose the advantage of looking up in a set then.

On the other hand, if you want to eliminate duplicates from a list, casting it to set can be a decent alternative even though there are also other choices for that in Python that may work better for your case.

There is a fair amount of more advanced ways to manage your code to make it more proficient and execute faster — from parallelism or simultaneousness to various tricks like designing smaller objects so that they fit in the cache layer of the heap memory as opposed to the main one. For certain assignments, you may have to use libraries that are actually intended to optimize those kinds of tasks.