takentui
Material

OOP in Python

Roadmap stageEngineering Python →

Object-oriented programming (OOP) - an approach in which a program is built from interacting objects that combine state and behavior. Objects are often instances of classes, but classes are not required to form a single application inheritance hierarchy.

OOP principles

Polymorphism

Polymorphism - the ability to work with data of different types through a single interface. There are two types of polymorphism:

Parametric polymorphism allows you to write code parameterized by a type, for example via TypeVar and generic types. Duck typing is closer to subtyping polymorphism in behavior: the code uses the interface supported by the object rather than requiring a concrete class. For example, we can create a function that will sum the length of all the sequences that we pass into it. And it doesn’t matter to us which classes the objects will belong to, they don’t have to be subtypes of the general class, the main thing is that they have a method implemented __len__. Ad-hoc polymorphism selects implementation depending on argument types. In Python it occurs, for example, in operator overloading through special methods and in functools.singledispatch; static overloads for type analyzers are described by typing.overload.

Encapsulation

Encapsulation - a language mechanism that allows you to combine data and methods that work with this data into a single object and hide implementation details from the user. Encapsulation in Python is implemented by defining class attributes and methods within a class. Hiding is an optional aspect of encapsulation that implies that the data of a class is hidden from direct access from outside the class.

Hiding in Python is not strict: the language uses naming conventions and name mangling rather than access modifiers that prohibit access to an attribute.

Inheritance

Inheritance - an OOP principle that lets us create a child class based on a parent class. The child class inherits the functionality of the parent class, and can override the parent's methods.

Python supports multiple inheritance. In a diamond hierarchy, the search order is determined by the full C3 linearization of MRO rather than a simple left-to-right rule. Illustration for the material “OOP in Python” Types of inheritance:

Single

Illustration for the material “OOP in Python”

Multi-level

Illustration for the material “OOP in Python”

Plural

Illustration for the material “OOP in Python”

Hierarchical

Illustration for the material “OOP in Python”

Hybrid

combines two or more types of inheritance Illustration for the material “OOP in Python”

By default, all user-defined classes in Python inherit from object. To find out whether one class is an inheritor (at any level, not just a direct descendant) of another, there is a function issubclass(child, parent).

Abstraction

Abstraction - this is the use of only those characteristics of an object that represent it with sufficient accuracy. We hide the part of the implementation that we don't need to interact with to make working with the object easier. In Python, abstraction is implemented using abstract classes from the ABC module.

Abstract classes

Abstract class is a class that is a template class for other classes. It contains abstract methods and is not intended for creating objects directly (if we try to create an instance of an abstract class, we will get an error). Abstract classes are needed to create a common interface for classes and to ensure that all child classes will have an implementation of each abstract method. If we try to create a class that inherits from an abstract class and do not implement all the abstract methods, we will get an error. The difference between an interface and an abstract class is that an interface does not contain an implementation, but an abstract class can, in addition to abstract methods, contain some implemented methods. Module abc provides a base class ABC, from which abstract classes and decorator must inherit @abstractmethod to create abstract methods.

from abc import ABC, abstractmethod

class Animal(ABC):
    def __init__(self, name, age):
        self.name = name
        self.age = age

    @abstractmethod
    def make_sound(self):
        pass

class Dog(Animal):
    def make_sound(self):
        return "Woof!"

class Cat(Animal):
    def make_sound(self):
        return "Meow!"

MRO (Method Resolution Order)

MRO is the search order for methods and attributes in a class and its base classes. Since Python 2.3 for new-style classes, and in Python 3 for all classes, it is used C3 linearization. This is not breadth-first traversal: C3 preserves the local order of the base classes and the monotonicity of the hierarchy. Cm. Python MRO HOWTO. MRO can be found using the function mro(). There may be situations where the class hierarchy we are trying to implement cannot be linearized. Then we will get an error TypeError: Cannot create a consistent method resolution order:

class X:
    pass

class Y(X):
    pass

class A(X, Y):
    pass

Composition

Composition — an object-oriented programming concept in which a class includes objects of another class as attributes. Instead of inheriting behavior, an object uses another class's functionality as part of its implementation. For example, we have the classes Car and Engine, Engine refers to Car, but is not a subspecies of it. That is, the connection is not is-a, as with inheritance (for example Sedan is a Car), but has-a: Car has an Engine.

class Engine:
    def start(self):
        print("Engine started")

class Car:
    def __init__(self):
        self.engine = Engine()  # Композиция - объект Car включает в себя объект Engine

    def drive(self):
        print("Car is moving")
        self.engine.start()

slots

Objects store properties and their values in a dictionary __dict__. Since a dictionary is a mutable structure, you can add and remove fields from a class on the fly, but this requires more resources. Parameter __slots__ in a class rigidly fixes a set of object fields. At the same time, in the class itself we can still declare any properties. __slots__ can remove separate __dict__ instances and reduce memory consumption, but does not provide a universal guarantee of acceleration. __dict__ may be inherited, added explicitly to slots, or reappeared in a subclass without __slots__.

class Person:
    __slots__ = ('name', 'age')

    def __init__(self, name, age):
        self.name = name
        self.age = age

Hagfish

Mixin (mix-in - “mixin”) is a small helper class that is added to the inheritance chain. They allow you to add new functionality to classes without changing their inheritance hierarchy. Mixins can be used to solve common problems that arise across different classes. For example, a mixin for logging or caching method results. It is customary to add the word to the names of mixins Mixin, since there is no mechanism for understanding whether this is a full-fledged class or mixin. Mixin is technically the most common class.

not very important

For cooperative multiple inheritance, class constructors must have compatible signatures, call super() and correctly pass arguments further - often through named parameters or **kwargs. "No parameters" is a possible design convention, not a Python rule.

super() function

Function super() returns a proxy for delegated lookup of attributes after the specified class in MRO. This is not an instance of the parent class, and the call itself super() does not call the method yet. For example, in our child class we have __init__, which takes one extra parameter than __init__ in the parent class. Then, in order not to duplicate the code, we call via super() __init__ in the parent class and pass arguments from the child class there:

class Parent:
    def __init__(self, name):
        self.name = name

class Child(Parent):
    def __init__(self, name, age):
        super().__init__(name)
        self.age = age

super(type, object_or_type=None) takes 2 arguments - a class and an object of this class. The class specifies which class in MRO to start searching for a method from. A class object specifies that the resulting method should be bound to that object. The two super() calls below are the same:

class Square(Rectangle):
    def __init__(self, width, height):
        super().__init__(width, height)

But for most cases a call is enough super() without parameters.

Public, protected and private

There are no forced access modifiers in Python public, protected and private, as in some other languages.

  • A name without a leading underscore is considered a public API.
  • Name _method by convention it is considered an internal implementation detail. The interpreter does not deny access to it either to external code or to descendants.
  • Name __method runs name mangling: inside the class definition it is converted to _ClassName__method. This helps avoid accidental name collisions in descendants, but does not provide security or make the attribute inaccessible from the outside.
class User:
    def __init__(self):
        self.__name = "John"


john = User()
print(john._User__name)

Name mangling - a mechanism in Python by which names of private properties of objects begin with two underscores like __method, are replaced by _Class__method

Classmethod and staticmethod

@classmethod is a method that gets a reference to a class cls as an implicit first argument, just like a regular instance method receives a reference to the instance self. It is used if we work in a method only with class attributes. @staticmethod – used to create a method that knows nothing about the class or instance through which it was called. It simply receives the arguments passed, without the implicit first argument self or cls. In fact, this is a regular function, defined for convenience within a class and which can be called through an object of the class or its instance. Can be useful for helper functions to avoid polluting the module namespace.

Getters and setters, property

Getters and setters allow you to keep an attribute's API stable and add evaluation or validation. They encapsulate implementation details, but are not a security mechanism in themselves. Decorator @property used to make a method a property. That is, we will use the syntax of a regular access to an object property, and under the hood we will call the corresponding getter or setter.

class Alphabet:
    def __init__(self, value):
        self._value = value

    # getting the values
    @property
    def value(self):
        print('Getting value')
        return self._value

    # setting the values
    @value.setter
    def value(self, value):
        print('Setting value to ' + value)
        self._value = value

    # deleting the values
    @value.deleter
    def value(self):
        print('Deleting value')
        del self._value

Decorator @property is syntactic sugar for the function property(fget=None, fset=None, fdel=None, doc=None), which implements handle protocol. Options fget, fset, fdel are responsible for the getter, setter and delimiter respectively.

Magic methods for attributes

__setattr__(self, key, value) - automatically called when the key property changes __getattribute__(self, item) - automatically called when a property with the name is received item __getattr__(self, item) - automatically called when a non-existent property named item is received __delattr__(self, item) - automatically called when a property named item is deleted (whether it exists or not)

Object creation process

The process of creating an object in Python is a sequential call of two methods:
__new__(cls, *args, **kwargs) called first and is directly responsible for creating an object, it takes a reference to the class whose object is being created. __new__ returns the created class object. This method is rarely overridden, usually when inheriting from immutable types or when implementing the Singleton pattern. __init__(self, *args, **kwargs) is responsible for initializing an already created object. He must return None; explicitly returning a different value results in TypeError. For example, we have a class Language:

class Language:
    def __init__(self, lang, year):
        self.lang = lang
        self.year = year

Then creating an object through a class call

language = Language('Python', 1991)

Will be equivalent to this code

language = object.__new__(Language)
language.__init__('Python', 1991)

Descriptors

Handles are Python objects that implement the handle protocol, which allows objects to be created with special behavior when accessed as attributes of other objects. The handle protocol consists of the following methods:

__get__(self, obj, owner=None) -> Any
__set__(self, obj, value) -> None
__delete__(self, obj) -> None
__set_name__(self, owner, name)

self is an instance of the descriptor
obj - the object to which the handle is attached.
owner - the class of the object to which the handle is attached. If the handle only implements __get__, then this is a non-data descriptor. If he also implements __set__ or __delete__, then this is the data descriptor. Handles involved in the chain attribute lookup. The handle object is created only once per property. This means that all objects with such a property share the same handle object. Therefore, the values ​​need to be stored not in the descriptor object:

def __set__(self, obj, value):
    self.value = value

and in the object on which the handle is called:

def __set__(self, obj, value) -> None:
    obj.__dict__[self.name] = value

Attribute self.name we set in the method __set_name__, which is called when a handle object is created. Here in the attribute name the value is rolled over 'number':

class OneDigitNumericValue():
    def __set_name__(self, owner, name):
        self.name = name

class Foo():
    number = OneDigitNumericValue()

Descriptors are used quite rarely. One of the use cases is the creation of lazy properties:

import time

class LazyProperty:
    def __init__(self, function):
        self.function = function
        self.name = function.__name__

    def __get__(self, obj, type=None) -> object:
        obj.__dict__[self.name] = self.function(obj)
        return obj.__dict__[self.name]

class DeepThought:
    @LazyProperty
    def meaning_of_life(self):
        time.sleep(3)
        return 42

my_deep_thought_instance = DeepThought()
print(my_deep_thought_instance.meaning_of_life)
print(my_deep_thought_instance.meaning_of_life)
print(my_deep_thought_instance.meaning_of_life)

# весь код выполнится за 3 секунды

This works through the attribute lookup order (lookup chain).

A descriptor can also replace repeated logic in multiple properties:

class Values:
    def __init__(self):
        self._value1 = 0
        self._value2 = 0

    @property
    def value1(self):
        return self._value1

    @value1.setter
    def value1(self, value):
        self._value1 = value if value % 2 == 0 else 0

    @property
    def value2(self):
        return self._value2

    @value2.setter
    def value2(self, value):
        self._value2 = value if value % 2 == 0 else 0

    # и еще много таких же property...

Instead, we can provide a descriptor:

class EvenNumber:
    def __set_name__(self, owner, name):
        self.name = name

    def __get__(self, obj, type=None) -> object:
        if obj is None:
            return self
        return obj.__dict__.get(self.name, 0)

    def __set__(self, obj, value) -> None:
        obj.__dict__[self.name] = (value if value % 2 == 0 else 0)

class Values:
    value1 = EvenNumber()
    value2 = EvenNumber()

Attribute lookup

When we access an attribute of an object using a dot, Python looks for that attribute in this order:

  1. First we will get the result returned by the method __get__ data descriptor with the name of the attribute you are looking for.
  2. If this fails, then we get the value from the dictionary __dict__ object by key with the name of the sought attribute.
  3. If this fails, then we will get the result returned by the method __get__ non-data descriptor with the name of the attribute you are looking for.
  4. If this fails, then we get the value from the dictionary __dict__ the class to which the object belongs, by the key with the name of the attribute being sought.
  5. If this fails, the attribute will be looked up in dictionaries __dict__ all classes from which the object class is inherited, according to MRO.
  6. If a normal search does not find an attribute, Python can call __getattr__. If it does not return a value, it will be thrown AttributeError.

Magic methods

Magical are called methods whose names begin and end with double underscores. They are magical because they are almost never called explicitly. They are called by built-in functions or syntactic constructs. For example, the function len() calls a method __len__() passed object. Method __add__(self, other) called automatically when adding by operator +. __add__ - addition + __sub__ - subtraction - __mul__ - multiplication * __truediv__ - division / __eq__ ==
__ne__ !=
__lt__ <
__le__ <=
__gt__ >
__ge__ >= When we override a method __eq__ in a class, then the method in it stops working by default __hash__, therefore, if it is needed, it must also be redefined. Function bool() for custom classes calls the method __bool__, if it is not defined, then it looks at __len__, if it is also not defined - returns True. __getitem__(self, item) - getting value by key item __setitem__(self, key, value) - record value value by key key __delitem__(self, key) - deleting an element by key key __repr__ is responsible for displaying an object within our system, which can be used to recreate the object. __str__ returns a less formal representation of the object that will be displayed to the user. If the method __str__ is not explicitly specified, it is called __repr__

Exceptions

Exceptions are thrown by the interpreter or if we manually write raise. There is a design for processing try except else finally.

try:
    risky_operation()
except (TypeError, ValueError) as error:
    handle_expected_error(error)
except LookupError as error:
    handle_lookup_error(error)
else:
    handle_success()
finally:
    cleanup()

Block finally executed when exiting try, including before the function actually returns. return inside finally suppresses a previously prepared result or exception, so is almost always a design error. It is necessary to handle the most specific type of exception possible, so as not to touch unnecessary ones and the behavior of the program is more predictable. The main class of exceptions is BaseException, is inherited from it Exception, and from Exception most of all exceptions Illustration for the material “OOP in Python” We can create our own custom exceptions; to do this we need to create a class that inherits from Exception or his descendant. It is customary to name exceptions so that their class name ends with the word Error.

Cexception chaining

Exception chaining is a technique for handling exceptions by re-throwing a caught exception after wrapping it in a new exception. The original exception is saved as a property of the new exception. When an exception is raised inside a block except, the old exception is stored in the data attribute __context__ and if the new exception is not handled, then information will be displayed that the new exception arose while processing the old one “During handling of the above exception, another exception occurred". You can also link exceptions into one chain or replace old ones with new ones. For this, the construction is used raise новое_исключение from старое_исключение, then the old exception is stored in the attribute __cause__ and the message “The above exception was the direct cause of the following exception". Or raise новое_исключение from None, then when outputting the old exception from the block except, will in fact be replaced by the new one (although the old exception is still stored in __context__). add exception group

Context manager

The context manager is a construct with, which provides access control to a certain resource. It performs some actions before gaining access to a resource and after working with it is completed. The context manager must implement two methods:

  • __enter__ (__aenter__ for async). Fires at the moment of entry and can return some value that can be obtained using the operator as:
with open('file.txt') as f:
    data = f.read()
  • __exit__ (__aexit__ for async) is triggered at the moment the block exits, including due to an exception. In this case, 3 values will be passed to the method (exception_type, exception_value, exception_traceback). If no exception occurs, then these parameters will be None. When handling exceptions in a method __exit__ what matters is what it returns.
  1. If the exception was handled normally, the method should return True.
  2. False value including None, does not create a new exception, but allows the original exception to propagate higher. If there was no exception, the return value is ignored. Besides magic methods, we can implement a context manager using a decorator @contextmanager from the built-in library contextlib
from contextlib import contextmanager

@contextmanager
def my_context_manager():
    # Код, выполняемый при входе в контекст (начало блока with)
    print("Вход в контекст")

    try:
        # Возвращаем значение, которое будет доступно в блоке with
        yield "Значение, доступное в блоке with"
    finally:
        # Код, выполняемый при выходе из контекста (окончание блока with)
        print("Выход из контекста")

Metaclasses

Metaclasses are a template for classes. The main purpose of metaclasses is to automatically modify a class at the time of creation. This is usually done for APIs, a typical example is Django ORM. We create a simple model class, and Django under the hood converts it into a more complex class:

class Person(models.Model):
  name = models.CharField(max_length=30)
  age = models.IntegerField()

Most often, metaclasses define a method __new__to, for example, add a new method to the class being created. Methods can also be defined __prepare__, __init__, __call__. An example of a built-in metaclass is type. To create your own metaclass, you need to inherit it from type. To create a class using a metaclass, you need to specify it when creating the class in the parameter metaclass.

class MyMeta(type):
    def __new__(cls, name, bases, dct):
        # Изменения в классе
        dct['new_method'] = lambda self: print("New method added!")
        return super().__new__(cls, name, bases, dct)

class MyClass(metaclass=MyMeta):
    def existing_method(self):
        print("Existing method.")

obj = MyClass()
obj.new_method()  # Выведет: New method added!
obj.existing_method()  # Выведет: Existing method.

By default, each user class is an instance of a class type. In the example above MyMeta - instance type, and MyClass - instance MyMeta. Metaclasses are rarely used in everyday programming, and most developers can get by without them.

How to create a class without keyword class?

Using the type function. In addition to returning the class to which the instance passed to it belongs, this function is also a built-in metaclass and can be used to create classes. It takes three arguments:

  1. name: This is a string containing the name of the class to create.
  2. bases (base classes): This is a tuple containing the base classes from which the generated class inherits. If the class does not inherit anything, you can pass an empty tuple ().
  3. dict (attribute dictionary): This is a dictionary containing the attributes and methods of the class being created. Dictionary keys are the names of attributes or methods, and values ​​are the attributes or functions themselves.

SOLID

SOLID is a set of 5 principles of object-oriented programming and design.

Single responsibility

Single responsibility principle: a module should be responsible for only one specific task, it should have only one reason for change. You need to combine code that changes for one reason and separate it from that that changes for another.

Example

Class User violates the principle of sole responsibility:

class User:
    def __init__(self, username, email):
        self.username = username
        self.email = email

    def save(self):
        # Логика сохранения пользователя в базе данных
        pass

    def send_email(self, message):
        # Логика отправки электронной почты пользователю
        pass

    def generate_report(self):
        # Логика генерации отчета пользователя
        pass

The class has several responsibilities. It is responsible for saving the user in the database, sending email and generating the report. As a result, the class becomes highly constrained and difficult to understand and support. Let's apply the single responsibility principle by dividing the class User into several separate classes with a single responsibility:

class User:
    def __init__(self, username, email):
        self.username = username
        self.email = email

    def save(self):
        # Логика сохранения пользователя в базе данных
        pass


class EmailSender:
    def send_email(self, user, message):
        # Логика отправки электронной почты пользователю
        pass


class ReportGenerator:
    def generate_report(self, user):
        # Логика генерации отчета пользователя
        pass

Open/closed

Open/closed principle: Modules should be open for expansion, but closed for modification. Modules should be designed so that they need to be changed as rarely as possible, and functionality can be expanded by creating new entities and combining them with old ones.

Example

For example, we are writing a messaging service. It accepts the text to be sent and a third-party API service for sending SMS or emails. A poorly designed service might look like this:

class SmsSender:
    def send_sms(self, text):
        pass

class EmailSender:
    def send_email(self, text):
        pass

class Notifier:
    def __init__(self, api: SmsSender | EmailSender):
        self.api = api

    def notify(self):
        message = 'Some user notification'

        if isinstance(self.api, SmsSender):
            self.api.send_sms(message)
        elif isinstance(self.api, EmailSender):
            self.api.send_email(message)

The problem with this code is that when adding a new type of third-party API - push notifications - we will have to change the existing code:

# предыдущие классы

# добавили новый API:
class PushSender:
    def send_push(self, text):
        pass

class Notifier:
    def __init__(self, api: SmsSender | PushSender | EmailSender):
        self.api = api

    def notify(self):
        message = 'Some user notification'

        if isinstance(self.api, SmsSender):
            self.api.send_sms(message)
        elif isinstance(self.api, EmailSender):
            self.api.send_email(message)
        elif isinstance(self.api, PushSender):  # это новый код, который
            self.api.send_push(message)         # пришлось добавить

OCP suggests not checking specific types, but using an abstraction that will allow you not to change the class code Notifier. To do this we will create an abstract class Sender, from which classes will inherit SmsSender, PushSender and EmailSender:

from abc import ABC, abstractmethod


class Sender(ABC):
    @abstractmethod
    def send_message(self, text: str) -> None:
        pass

class SmsSender(Sender):
    def send_message(self, text):
        # то, что раньше было внутри метода send_sms()
        pass

class EmailSender(Sender):
    def send_message(self, text):
        # то, что раньше было внутри метода send_email()
        pass

class PushSender(Sender):
    def send_message(self, text):
        # то, что раньше было внутри метода send_push()
        pass

class Notifier:
    def __init__(self, api: Sender):
        self.api = api

    def notify(self):
        message = 'Some user notification'

        self.api.send_message(message)

Liskov substitution

Barbara Liskov's substitution principle: if a function can work with a base class, then it should be able to work with a descendant of this class. It's easier to follow LSPs if you don't build large and deep hierarchies. Usually, an interface and its implementations are sufficient to connect modules.

Example

Let's say we have a base class of a bank account Account, which has three methods: view account balance, top up and pay. We need to write two more classes: salary account SalaryAccount and deposit account DepositAccount, while the salary account must support all operations presented in the base class, and the deposit account must not support payments. If in the program code everywhere we used the class Account replace with its descendant class SalaryAccount, then the program will continue to work normally, since in the class SalaryAccount all operations available in the class are available Account. If we try to do this with the class DepositAccount, that is, we replace the base class Account to its successor class DepositAccount, then the program will start to work incorrectly, since when calling the payment method payment() an exception will be thrown. Thus, a violation of Barbara Liskov's substitution principle occurred.

Interface segregation

Interface segregation principle: Software entities should not depend on interfaces that they do not use. A descendant class should not drag down methods of its parent class that it does not use. Most often this is achieved by fragmenting the interface.

Example

For example, we have an abstract class Vehicle with methods go() and fly(), from which they inherit Aircraft and Car. An airplane can both drive and fly, but a car can only drive, that is, it has to implement a method fly()which it doesn't use is an ISP violation: Illustration for the material “OOP in Python” The solution is to split the interface Vehicle to smaller interfaces Movable and Flyable: Illustration for the material “OOP in Python”

Dependency inversion

Low level modules contain utilitarian functionality: accessing the database, queries to the server, rendering DOM elements on the page. High level modules contain complex, more abstract business logic. They are abstract enough that they can be reused in different projects: user authorization, form validation, sending notifications. Dependency inversion principle: high-level modules should not depend on low-level ones; classes should depend on abstractions, not concrete implementations. You should not create objects in a class/function that uses these objects. It’s better to hand over ready-made ones. This reduces coupling between classes, makes them more flexible and more testable (due to the ability to create stubs that implement the desired interface). The principle of dependency inversion reduces the coupling of modules and allows you to design a system so that modules can be replaced with others.

Example

Let's take a class that has a service for sending notifications:

from abc import ABC, abstractmethod
from typing import Any


Report = Any


class ReportSender(ABC):
    @abstractmethod
    def send(self, report: Report):
        pass

class TelegramReportSender(ReportSender):
    def send(self, report: Report):
        print("Отправили отчет в телегу")

class EmailReportSender(ReportSender):
    def send(self, report: Report):
        print("Отправили отчет на мыло")

class UserService:
    def __init__(self, username: str):
        self._username = username
        self._report_sender = TelegramReportSender()

In this class we depend on a specific implementation ReportSender'a - from TelegramReportSender. To prevent this from happening, we can pass implementations ReportSender' and into the constructor:

class UserService:
    def __init__(self, username: str, report_sender: ReportSender):
        self._username = username
        self._report_sender = report_sender

user_service_with_email = UserService("kiriharu", EmailReportSender())
user_service_with_tg = UserService("kiriharu", TelegramReportSender())

Official sources

  • Python data model
  • abc — abstract base classes
  • Descriptor HOWTO
  • MRO HOWTO