Raven Note

Setting up Tailscale and configuring HTTPS on Synology NAS: A practical guide

Wed, 31 Dec 2025 00:00:00 GMT

To enhance Synology NAS security (especially EOL models), and enable remote access without a public IP address, Tailscale can be used to build a secure private network.

Setting up Tailscale#

For Synology NAS models outside the mainland China market, Tailscale is available directly through Package Center. Otherwise, download the appropriate SPK package for your DSM version and CPU architecture from Tailscale Packages, and then manually install it in Package Center (if you’re unsure about the architecture, check Synology and SynoCommunity Package Architectures).

You can automate updates with a scheduled task. In DSM, open Control Panel, find Scheduled Task, and create a new scheduled task with a user-defined script. Set the running user to root, schedule it to run once a month, and use the following script:

1
tailscale update --yes

NOTE
Per the official recommendation, Tailscale should be kept up to date automatically via a scheduled task even when installed through Package Center

On DSM 7, stricter restrictions on package behavior mean Tailscale doesn’t have permission to create TUN devices, so by default external devices can connect to the NAS, but applications running on the NAS itself cannot initiate network connections through Tailscale. If you need that, open Control Panel in DSM, find Scheduled Task, create a new triggered task, select Boot as the event, set the user to root, and use the following script:

1
/var/packages/Tailscale/target/bin/tailscale configure-host; synosystemctl restart pkgctl-Tailscale.service

Make sure you’re running Tailscale v1.22.2 or later

You’ll need to run the command above again after upgrading the Tailscale package

Setting up HTTPS and automating it#

Open the Tailscale admin console, go to the DNS tab, find HTTPS Certificates, and click Enable HTTPS.

Make sure MagicDNS is enabled before turning this on

Once enabled, machine names and Tailnet DNS names become publicly visible (this doesn’t affect security)

Next, open Control Panel in DSM, go to Terminal & SNMP, and enable SSH.

Once connected to the NAS over SSH, switch to the root account (a small shortcut) and run:

1
tailscale cert [Your Domain]

This command generates two files. Find a way to get them onto your computer (there are plenty of options — e.g., move them to a shared folder and download them via File Station, or use SCP).

Then open Control Panel in DSM, go to Security, switch to the Certificate tab, click New, and follow the wizard to import the certificate. Set the description to Tailscale and leave the intermediate certificate blank.

Make sure the description is exactly Tailscale — otherwise the automation script won’t be able to locate the certificate path later.

The automation script has only been tested on DSM 7. Before relying on it, verify that the jq command resolves correctly when run as root. If it doesn’t, adjust the jq query based on the structure of /usr/syno/etc/certificate/_archive/INFO, or pin down the fixed certificate directory by comparing sha256 hashes (you can quickly list the sha256 of every cert.pem — i.e., the sha256 of the crt files — with find /usr/syno/etc/certificate/_archive -type f -name cert.pem -exec sha256sum {} \;).

If you modify the script to use a fixed directory, be aware that the path changes whenever you delete and re-import the certificate — remember to update it then.

Once the import is done, don’t forget to go to Settings and switch the certificates used by the relevant services to the Tailscale certificate you just imported. HTTPS is actually fully configured at this point — but every time the certificate expires, you’d have to repeat the whole process manually, which is a pain, especially since certificate validity is being shortened to 45 days starting in 2028.

Write the following script, remembering to replace [Your Domain] with your full Tailscale domain:

1
#!/bin/bash
2
DOMAIN="[Your Domain]"
3
CERT_DIR=$(jq -r --arg desc "Tailscale" 'to_entries[] | select(.value.desc == $desc) | .key' /usr/syno/etc/certificate/_archive/INFO)
4
CERT_ARCHIVE_PATH="/usr/syno/etc/certificate/_archive/${CERT_DIR}"
5

6
if [ -f "${CERT_ARCHIVE_PATH}/cert.pem" ]; then
7
    EXPIRY_DATE=$(openssl x509 -enddate -noout -in "${CERT_ARCHIVE_PATH}/cert.pem" | cut -d= -f2)
8
    EXPIRY_EPOCH=$(date -d "${EXPIRY_DATE}" +%s)
9
    CURRENT_EPOCH=$(date +%s)
10
    DAYS_REMAINING=$(( (EXPIRY_EPOCH - CURRENT_EPOCH) / 86400 ))
11
else
12
    exit 1
13
fi
14

15
if [ ${DAYS_REMAINING} -gt 14 ]; then
16
    exit 0
17
fi
18

19
tailscale cert "${DOMAIN}"
20

21
if [ ! -f "${DOMAIN}.crt" ]; then
22
    exit 1
23
fi
24

25
cp "${DOMAIN}.crt" "${CERT_ARCHIVE_PATH}/cert.pem"
26
mv "${DOMAIN}.key" "${CERT_ARCHIVE_PATH}/privkey.pem"
27
mv "${DOMAIN}.crt" "${CERT_ARCHIVE_PATH}/fullchain.pem"
28

29
chmod 400 "${CERT_ARCHIVE_PATH}/cert.pem"
30
chmod 400 "${CERT_ARCHIVE_PATH}/privkey.pem"
31
chmod 400 "${CERT_ARCHIVE_PATH}/fullchain.pem"
32

33
chown root:root "${CERT_ARCHIVE_PATH}/cert.pem"
34
chown root:root "${CERT_ARCHIVE_PATH}/privkey.pem"
35
chown root:root "${CERT_ARCHIVE_PATH}/fullchain.pem"
36

37
synosystemctl restart nginx

Save the content above as refresh_tailscale_cert.sh and place it in any shared folder. Then open Control Panel in DSM, find Scheduled Task, and create a new scheduled task with a user-defined script. Set the running user to root, schedule it to run weekly, and use the following script:

1
cd /[Your Volume]/[Your Shared Folder]
2
bash refresh_tailscale_cert.sh

If you’re not sure what /[Your Volume]/[Your Shared Folder] should be, right-click your script in DSM and check its properties.

References#

Issue Synology Let’s Encrypt Cert by acme.sh docker

Python from Beginner to Advanced: Classes, Objects, and Object-Oriented Programming

Wed, 18 Jun 2025 00:00:00 GMT

OOP: A Prologue#

When requirements grow complex, programming purely in a procedural style starts to hurt code readability and maintainability. Suppose you’re developing a library management program. At first, you might record book information in the most straightforward way possible:

1
book1_title = "Python Crash Course"
2
book1_author = "Eric Matthes"
3
book1_isbn = "978-7115613639"
4
book1_stock = 5
5

6
book2_title = "Fluent Python"
7
book2_author = "Luciano Ramalho"
8
book2_isbn = "978-1492056355"
9
book2_stock = 3

Before long you realize that every new book means redefining a whole pile of variables. Then a lightbulb goes off: pull each book’s data out and store it in a list:

1
books = [
2
    ["Python Crash Course", "Eric Matthes", "978-7115613639", 5],
3
    ["Fluent Python", "Luciano Ramalho", "978-1492056355", 3]
4
]

But now every lookup relies on indexes like books[0][0] or books[2][3], which is hardly intuitive — read this code in isolation and you have no idea what those indexes refer to. Luckily, you remember that Python has something called a dictionary:

1
books = [
2
    {
3
        "title": "Python Crash Course",
4
        "author": "Eric Matthes",
5
        "isbn": "978-7115613639",
6
        "stock": 5,
7
    },
8
    {
9
        "title": "Fluent Python",
10
        "author": "Luciano Ramalho",
11
        "isbn": "978-1492056355",
12
        "stock": 3,
13
    },
14
]

OK, that feels good enough, so you start writing the features:

1
def borrow(book: dict) -> None:
2
    if book["stock"] > 0:
3
        book["stock"] -= 1
4
        print(f"'{book['title']}' borrowed successfully! Remaining stock: {book['stock']}")
5
    else:
6
        print("Insufficient stock!")
7

8

9
def show_info(book: dict) -> None:
10
    print(
11
        f'"{book["title"]}" by {book["author"]} (ISBN: {book["isbn"]}), stock: {book["stock"]}'
12
    )
13

14

15
def find_book(isbn: str) -> dict | None:
16
    for book in books:
17
        if book["isbn"] == isbn:
18
            return book
19
    return None
20

21

22
book = find_book("978-1492056355")
23
if book:
24
    show_info(book)
25
    borrow(book)
26
else:
27
    print("Book not found!")

Not bad — you’ve taken your first step. But a mature library management system doesn’t just manage books. It also has to record who borrowed which book, track each book’s loan history including borrow and return dates, calculate fines for overdue books, and support queries about any user’s borrowing status. If you keep going procedural, you’ll find yourself adding a user dict that nests a list of dicts for borrowed books, while each book dict nests a list of who borrowed it plus a loan history. Your code ends up crammed with loops, conditionals, and nested lookups — touch one thing and everything else moves. The code runs, sure, but readability and maintainability take a serious hit.

At this point you might be thinking: wouldn’t it be nice to bundle data and its related operations together? That’s exactly when classes step onto the stage.

Conventions#

This article targets Python 3.12; any feature introduced after Python 3.12 will be explicitly noted.
For freshness, every reference is labeled with its year.
For freshness, any reference older than two years must be verified by hands-on testing.
Code blocks default to the contents of main.py.
Comments starting with Output: show what that line prints.
Comments starting with Raise: show the exception that line will raise.
This article assumes you’re already comfortable with basic procedural Python.

Basic Concepts#

In case you haven’t come across these concepts yet, let’s briefly go over some fundamentals before we start.

Magic Methods#

Every method surrounded by double underscores is a magic method, the most common being __init__(). We won’t dig into magic methods here — that comes in later chapters. One thing to keep in mind: never define your own methods with names that start and end with double underscores. They may not have a special meaning today, but there’s no guarantee that will stay true.

Decorators#

This is also covered in a later chapter. In short, a decorator modifies the behavior of a function or method, and it’s usually written as @xxxx.

Namespaces#

Say you’re looking for a friend named Zhang Wei — but there are 294,282 people named Zhang Wei in the country (2019). How do you know which Zhang Wei you mean? You need more than just the name: which city they live in, which company they work for, and so on.

Similarly, if methods or variables with the same name are defined in multiple places, the Python interpreter can’t tell which one you mean. That’s where the concept of namespaces comes in. A namespace groups names: the same name in different namespaces points to different objects. When you write code, you just prefix the method or variable name with its module name, and the interpreter knows exactly which one to call.

Here’s an example from computer systems: a folder (directory) can contain many folders. Within a single folder, no two files can share the same name, but files in different folders can have the same name.

(Image source: Runoob Tutorial)

Here’s a code example:

1
def echo():
2
    return "from module 1"

1
def echo():
2
    return "from module 2"

1
import module1
2
import module2
3
print(module1.echo())  # Output: from module 1
4
print(module2.echo())  # Output: from module 2

In Python, the . operator accesses an object’s attributes and methods. For instance, in module1.echo(), module1 is a module object and echo is one of its methods, invoked via .. With module1.echo() and module2.echo(), we’ve explicitly told the interpreter which module’s method to call.

Scope and Closures#

Now let’s talk about scope in Python. As the name suggests, scope is the region where a name is effective — in other words, “where this name can be used”. Python’s scoping follows the LEGB rule, which looks up a variable in this order:

Local: the local scope of the current function.
Enclosing: the scope of the enclosing function that contains the current one (if nested functions are involved).
Global: the current module’s global scope.
Builtin: Python’s built-in scope.

The diagram below makes it more intuitive:

(Image source: Runoob Tutorial)

Lookup proceeds one level at a time from the innermost scope outward, and an inner variable with the same name shadows the outer one. For example:

1
x = "global variable"
2
def main():
3
    x = "local variable"
4
    print(x)
5

6
main()  # Output: local variable
7
print(x)  # Output: global variable

Every time a function is called, it creates its own local scope. Arguments passed in are copied into the function’s parameters, which are only accessible inside the function’s local scope. Once the function finishes, its local variables are released from memory by the garbage collector:

1
def add(a, b):
2
    print(f"Inside the function: a = {a}")
3
    return a + b
4

5

6
result1 = add(2, 3)  # Output: Inside the function: a = 2
7
result2 = add(5, 7)  # Output: Inside the function: a = 5
8

9
print(a)  # Raise: NameError: name 'a' is not defined

So you could say scope has a kind of “enclosure”. You may have already realized that namespaces are an abstraction of scopes: they exploit scope’s ability to “seal names inside” in order to achieve grouping.

When you define a function inside another function, the inner function belongs to the outer function’s local scope:

1
def outer():
2
    print("Outer function executed")
3

4
    def inner():
5
        print("Inner function executed")
6

7
    inner()
8

9
outer()
10
inner()  # Raise: NameError: name 'inner' is not defined.

Calling it from inside the outer function’s local scope works fine, but trying to call it from the global scope raises a NameError, because inner isn’t defined there.

However, there is a way to access the function from outside its local scope: return the inner function object instead of calling it, and assign the returned function to a variable:

1
def outer():
2
    def inner():
3
        return "Inner function called"
4

5
    return inner
6

7
func = outer()
8
print(func())   # Output: Inner function called

The outer function has already finished running, so its local scope should have been destroyed — why does inner still work? Because the Python interpreter keeps any object in memory as long as there’s still a reference to it.

What if the inner function references a variable from the outer function’s scope? Here’s an example:

1
def power(exponent):
2
    def inner(base):
3
        return base ** exponent
4
    return inner
5

6
power_of_two = power(2)
7
power_of_three = power(3)
8

9
print(power_of_two(5))  # Output: 25
10
print(power_of_three(5))  # Output: 125

Theoretically, when power finishes executing, its local scope — including local variables like exponent — should be gone. But at runtime, power_of_two still accesses exponent and gets the right answer. That’s because Python handles this case specially. Variables defined in an enclosing scope rather than the function’s own local scope are called non-local variables. When an inner function references variables from an outer function, Python stores those free variables (variables used inside a function but not defined there) in a special attribute, __closure__. Such variables — exponent in this example — are called non-local free variables. That way they stay in memory even after the enclosing function’s local scope is destroyed. A function that carries its defining environment around with it like this is called a closure.

Even though a closure can read non-local variables, trying to assign to one directly just creates a local variable — it won’t modify the non-local one. That’s where nonlocal comes in. It tells the interpreter that a variable refers to a name already bound in the nearest enclosing scope, rather than creating a new local variable:

1
def main():
2
    x = 1
3
    def a():
4
        nonlocal x
5
        x = 2
6
    def b():
7
        x += 3  # Raise: UnboundLocalError: cannot access local variable 'x' where it is not associated with a value
8
    print(x)  # Output: 1
9
    a()
10
    print(x)  # Output: 2
11
    b()
12
    print(x)
13

14
main()

You might wonder why x += 3 raises UnboundLocalError. Because x += 3 is equivalent to x = x + 3; Python first needs to read the value of x to do the addition, but since direct assignment only creates a local variable, x is marked as local — and a local x was never assigned, so the interpreter raises UnboundLocalError.

Of course, if you don’t rebind the variable (for example, you only mutate a mutable object), you can use it directly:

1
def main():
2
    x = []
3

4
    def a():
5
        x.append(1)
6

7
    print(x)  # Output: []
8
    a()
9
    print(x)  # Output: [1]
10

11

12
main()

If you want to modify a global-scope variable instead, use global:

1
x = 1
2
def a():
3
    global x
4
    x = 2
5

6
print(x)  # Output: 1
7
a()
8
print(x)  # Output: 2

A First Look at Classes and Objects#

At the heart of object-oriented programming is abstraction — distilling things from the real world into objects, and using objects as the basic unit that encapsulates data and behavior to build programs.

Think about how you interact with everyday objects. You watch TV using the remote for power, volume, and channels — you never need to understand how the circuits inside work. OOP works the same way: related data and the methods that operate on that data are packaged together into a class. Callers only need to know what interfaces are available; they don’t care about the implementation details.

We use classes to create objects — this is called instantiation, and the objects produced are called instances of the class. Many languages describe classes as blueprints. In Python, though, a class is less like a blueprint and more like a factory, with instances as the products rolling off the line. Normally — unless you have a specific reason to do otherwise — you work with the products (instances), not the factory (class) itself.

In Python, the class keyword defines a class. For instance, let’s define a Book class:

1
class Book:
2
    pass

To instantiate it, just call it:

1
my_book = Book()

This creates a new instance of Book and assigns it to the local variable my_book. You can use the magic attribute __class__ to find out which class an instance belongs to, and the magic attribute __name__ to get the class’s name.

1
print(x.__class__)  # Output: <class '__main__.Book'>
2
print(x.__class__.__name__)  # Output: Book

A factory that produces nothing isn’t much use — we want products with real information. For instance, every book should have its own title, author, and ISBN:

1
class Book:
2
    def __init__(self, title, author, isbn, stock):
3
        self.title = title
4
        self.author = author
5
        self.isbn = isbn
6
        self.stock = stock
7

8
python_crash_course = Book("Python Crash Course", "Eric Matthes", "978-7115613639", 5)
9
fluent_python = Book("Fluent Python", "Luciano Ramalho", "978-1492056355", 3)
10

11
print(python_crash_course.title)  # Output: Python Crash Course
12
print(fluent_python.title)  # Output: Fluent Python

__init__() is a magic method that’s called automatically when a new instance is created, saving you from calling it manually.

Every instance has its own independent copy of the data. python_crash_course’s title and fluent_python’s title don’t affect each other. Attributes bound to a specific instance are called instance attributes. Usually you can inspect all the attributes of an instance created from a custom class via __dict__:

1
print(python_crash_course.__dict__)  # Output: {'title': 'Python Crash Course', 'author': 'Eric Matthes', 'isbn': '978-7115613639', 'stock': 5}
2
print(fluent_python.__dict__)  # Output: {'title': 'Fluent Python', 'author': 'Luciano Ramalho', 'isbn': '978-1492056355', 'stock': 3}

Functions defined in a class are called methods. A method that needs to access or modify the instance’s own attributes should be an instance method. The first parameter of an instance method must receive the instance object itself; by convention, this parameter is named self. Let’s add a method to Book that displays book information:

1
class Book:
2
    def __init__(self, title, author, isbn, stock):
3
        self.title = title
4
        self.author = author
5
        self.isbn = isbn
6
        self.stock = stock
7

8
    def get_book_info(self):
9
        return f'"{self.title}" by {self.author} (ISBN: {self.isbn}), stock: {self.stock}'
10

11

12
fluent_python = Book("Fluent Python", "Luciano Ramalho", "978-1492056355", 3)
13
print(fluent_python.get_book_info())  # Output: "Fluent Python" by Luciano Ramalho (ISBN: 978-1492056355), stock: 3

Sometimes an attribute or behavior belongs to the whole class rather than to any particular instance. For example, a library needs to know how many distinct titles it holds. That information doesn’t belong to any single book — it belongs to the Book class as a whole. Attributes shared by all instances are called class attributes.

Correspondingly, a method that operates on class attributes rather than instance attributes is usually defined as a class method. Class methods are marked with the @classmethod decorator, and their first parameter must receive the class object itself; by convention, this parameter is named cls:

1
class Book:
2
    _all_isbns = set()
3

4
    def __init__(self, title, author, isbn, stock):
5
        self.title = title
6
        self.author = author
7
        self.isbn = isbn
8
        self.stock = stock
9
        self._add_isbn(isbn)
10

11
    def get_book_info(self):
12
        return f'"{self.title}" by {self.author}, stock: {self.stock}'
13

14
    @classmethod
15
    def get_store_info(cls):
16
        return f"Welcome to the library! We currently hold {len(cls._all_isbns)} distinct titles."
17

18
    @classmethod
19
    def _add_isbn(cls, isbn: str):
20
        cls._all_isbns.add(isbn)
21

22
python_crash_course = Book("Python Crash Course", "Eric Matthes", "978-7115613639", 5)
23
fluent_python = Book("Fluent Python", "Luciano Ramalho", "978-1492056355", 3)
24

25

26
print(Book.get_store_info())  # Output: Welcome to the library! We currently hold 2 distinct titles.
27
print(fluent_python.get_store_info())  # Output: Welcome to the library! We currently hold 2 distinct titles.

Class methods can be called directly on the class. Calling them through an instance also works and produces the same result, but it’s not recommended.

self and cls are just conventions — what matters is that they occupy the first parameter position. As long as they come first, the name doesn’t affect whether the code runs. You could even swap self and cls on a whim:

1
class Book:
2
    _all_isbns = set()
3

4
    def __init__(cls, title, author, isbn, stock):
5
        cls.title = title
6
        cls.author = author
7
        cls.isbn = isbn
8
        cls.stock = stock
9
        cls._add_isbn(isbn)
10

11
    @classmethod
12
    def get_store_info(self):
13
        return f"Welcome to the library! We currently hold {len(self._all_isbns)} distinct titles."
14

15
    @classmethod
16
    def _add_isbn(self, isbn: str):
17
        self._all_isbns.add(isbn)
18

19
    def get_book_info(cls):
20
        return f'"{cls.title}" by {cls.author} (ISBN: {cls.isbn}), stock: {cls.stock}'

It would run, sure — but if anyone else ever takes over the project, you’d better hope you can run even faster. That’s the whole point of conventions: the next person to touch the code shouldn’t have to pause and puzzle out what each parameter is for.

When a method needs neither instance attributes nor class attributes but is still functionally tied to the class, use a static method. For example, let’s add a method that validates ISBNs:

1
class Book:
2
    _all_isbns = set()
3

4
    def __init__(self, title, author, isbn, stock):
5
        if not self._is_valid_isbn(isbn):
6
            raise ValueError("Invalid ISBN")
7
        self.title = title
8
        self.author = author
9
        self.isbn = isbn
10
        self.stock = stock
11
        self._add_isbn(isbn)
12

13
    def get_book_info(self):
14
        return f'"{self.title}" by {self.author}, stock: {self.stock}'
15

16
    @classmethod
17
    def get_store_info(cls):
18
        return f"Welcome to the library! We currently hold {len(cls._all_isbns)} distinct titles."
19

20
    @classmethod
21
    def _add_isbn(cls, isbn: str):
22
        cls._all_isbns.add(isbn)
23

24
    @staticmethod
25
    def _is_valid_isbn(isbn: str) -> bool:
26
        clean = isbn.replace("-", "").replace(" ", "")
27
        if len(clean) != 13 or not clean.isdigit():
28
            return False
29

30
        total = sum(int(d) * (1 if i % 2 == 0 else 3) for i, d in enumerate(clean))
31
        return total % 10 == 0
32

33
fluent_python = Book("Fluent Python", "Luciano Ramalho", "123-4567890123", 3)  # Raise: ValueError: Invalid ISBN

Class attributes can be assigned via class methods or directly on the class, but not on an instance. An instance can read a class attribute, but trying to assign to it creates a new instance attribute instead of modifying the class attribute (mutating a mutable object is the exception):

1
class Book:
2
    _all_isbns = set()
3
    test = 1
4

5
    def __init__(self, title, author, isbn, stock):
6
        self.title = title
7
        self.author = author
8
        self.isbn = isbn
9
        self.stock = stock
10
        self._add_isbn(isbn)
11

12
    @classmethod
13
    def _add_isbn(cls, isbn: str):
14
        cls._all_isbns.add(isbn)
15

16
python_crash_course = Book("Python Crash Course", "Eric Matthes", "978-7115613639", 5)
17
print(Book.__dict__) # {'__module__': '__main__', '_all_isbns': {'978-7115613639'}, 'test': 1, '__init__': <function Book.__init__ at 0x0000020EF37AD1C0>, '_add_isbn': <classmethod(<function Book._add_isbn at 0x0000020EF37AD260>)>, '__dict__': <attribute '__dict__' of 'Book' objects>, '__weakref__': <attribute '__weakref__' of 'Book' objects>, '__doc__': None}
18
print(python_crash_course.__dict__) # {'title': 'Python Crash Course', 'author': 'Eric Matthes', 'isbn': '978-7115613639', 'stock': 5}
19
python_crash_course._all_isbns.add("978-1492056355")
20
python_crash_course.test = 123
21
print(Book.__dict__) # {'__module__': '__main__', '_all_isbns': {'978-7115613639', '978-1492056355'}, 'test': 1, '__init__': <function Book.__init__ at 0x0000020EF37AD1C0>, '_add_isbn': <classmethod(<function Book._add_isbn at 0x0000020EF37AD260>)>, '__dict__': <attribute '__dict__' of 'Book' objects>, '__weakref__': <attribute '__weakref__' of 'Book' objects>, '__doc__': None}
22
print(python_crash_course.__dict__) # {'title': 'Python Crash Course', 'author': 'Eric Matthes', 'isbn': '978-7115613639', 'stock': 5, 'test': 123}

Reuse Through Inheritance#

When you write a class, you don’t always have to start from scratch. If the class you want is a specialized version of an existing class, use inheritance. When one class inherits from another, it automatically gets all of the parent’s attributes and methods. The original class is called the parent class, and the new class is the child class. A child class not only inherits everything from its parent — it can also define its own attributes and methods.

Say we have a Vehicle class and we want to write a Car class — inheritance fits perfectly. The syntax is straightforward: put the base class’s name in parentheses when defining the class.

1
class Vehicle:
2
    def __init__(self, brand):
3
        self.brand = brand
4

5
    def drive(self):
6
        print("The vehicle is driving")
7

8
class Car(Vehicle):
9
    def __init__(self, brand):
10
        super().__init__(brand)
11

12
    def honk(self):
13
        print("Beep beep")

super() lets us call methods on the parent class. That line invokes Vehicle’s __init__(), so Car instances get all the attributes defined there.

A child class can also provide a new implementation of a method with the same name as the parent’s — this is called overriding. It lets the child class tailor inherited behavior to its own needs:

1
class Car(Vehicle):
2
    def __init__(self, brand):
3
        super().__init__(brand)
4

5
    def drive(self):
6
        print(f"{self.brand} car is driving")
7

8
    def honk(self):
9
        print("Beep beep")

Suppose you accidentally spell drive as driv — a method that doesn’t exist in the parent. When you call drive on the child, it would silently fall back to the parent’s drive. To let type checkers catch this mistake, annotate the method with @override to make the override relationship explicit:

1
from typing import override
2

3
class Vehicle:
4
    def __init__(self, brand):
5
        self.brand = brand
6

7
    def drive(self):
8
        print("The vehicle is driving")
9

10

11
class Car(Vehicle):
12
    def __init__(self, brand):
13
        super().__init__(brand)
14

15
    @override
16
    def driv(self):
17
        print(f"{self.brand} car is driving")
18

19
    def honk(self):
20
        print("Beep beep")

Python allows a class to inherit from multiple parent classes at once. For example:

1
class Radio:
2
    def play_music(self, name):
3
        print(f"Radio is playing: {name}")
4

5
class MusicCar(Car, Radio):
6
    pass
7

8
my_music_car = MusicCar("Unknown")
9
my_music_car.drive()  # from Car
10
my_music_car.play_music("Big Fish - Zhou Shen")  # from Radio

When a class has multiple parents that may define methods with the same name, Python follows a well-defined, predictable order to look up methods (including for super()). That order is the MRO. Generally you don’t need to know how Python computes it — you can just inspect the result with __mro__:

1
print(MusicCar.__mro__) # Output: (<class '__main__.MusicCar'>, <class '__main__.Car'>, <class '__main__.Vehicle'>, <class '__main__.Radio'>, <class 'object'>)

From the output, the order is MusicCar -> Car -> Vehicle -> Radio -> object.

You can use isinstance() to check whether an object is an instance of a class or one of its subclasses:

1
class Vehicle:
2
    def __init__(self, brand):
3
        self.brand = brand
4

5
    def drive(self):
6
        print("The vehicle is driving")
7

8

9
class Car(Vehicle):
10
    def __init__(self, brand):
11
        super().__init__(brand)
12

13
    def drive(self):
14
        print(f"{self.brand} car is driving")
15

16

17
class Bike(Vehicle):
18
    def __init__(self, brand):
19
        super().__init__(brand)
20

21
    def drive(self):
22
        print(f"{self.brand} bike is driving")
23

24
my_bike = Bike("Unknown")
25
my_car = Car("Unknown")
26

27
print(isinstance(my_bike, Bike))  # Output: True
28
print(isinstance(my_bike, Vehicle))  # Output: True
29
print(isinstance(my_car, Vehicle))  # Output: True
30
print(isinstance(my_car, Bike))  # Output: False

Do We Really Need Inheritance?#

Inheritance models an “is-a” relationship — a car is a vehicle, for instance. But we also run into “has-a” relationships: the MusicCar in the earlier example has a radio. In that case, instead of inheriting, it’s usually better to compose: use an instance of the other class as an attribute. Here’s a concrete example:

1
class Radio:
2
    def play_music(self, name):
3
        print(f"Radio is playing: {name}")
4

5
class Car:
6
    def __init__(self, brand):
7
        self.brand = brand
8
        self.radio = Radio()
9

10
    def drive(self):
11
        print(f"{self.brand} car is driving")
12

13
    def play_music(self, name):
14
        self.radio.play_music(name)
15

16
my_car = Car("Unknown")
17
my_car.drive()  # Output: Unknown car is driving
18
my_car.play_music("Big Fish - Zhou Shen") # Output: Radio is playing: Big Fish - Zhou Shen

In practice, whether to use inheritance or composition comes down to whether the two classes have a clear hierarchical relationship. If they don’t, composition is the better approach.

Customizing Class Behavior#

Sometimes we need to customize a class’s behavior. We can do this with magic methods (in most cases) or metaclasses (in rare cases).

With Magic Methods#

Customizing Instantiation with `new`#

The magic method we use most is __init__(). But __init__() isn’t the first thing called — __new__() runs before it. By default, __new__() creates the instance and passes it as the first argument to __init__(), which receives the new instance and returns it. By overriding __init__() we can automatically add attributes to newly created instances; by overriding __new__() we can customize the instance creation process itself. For example, here’s the singleton pattern:

1
class A:
2
    _instance = None
3
    _initialized = False
4

5
    def __new__(cls, *args, **kwargs):
6
        if not cls._instance:
7
            cls._instance = super().__new__(cls)
8
        return cls._instance
9

10
    def __init__(self, test):
11
        if not self._initialized:
12
            self._initialized = True
13
            self.test = test
14

15
a1 = A(test=10)
16
a2 = A(test=20)
17

18
print(a1 is a2)  # Output: True
19
print(a2.test)  # Output: 10

Because we’ve overridden __new__(), we must explicitly call the parent’s instantiation logic — that is, super().__new__(cls) (the parent is object, the root of all classes). We also must return an instance; otherwise the rest of the machinery never runs and instantiation is cut short. And if we returned an instance of some other class here, that’s the instance the rest of the process would use.

Also, since __init__() runs on every instantiation, we need to limit how many times it takes effect — otherwise later instantiations would overwrite the existing singleton’s state, unless that’s the behavior you want.

There’s one more magic method tied to an instance’s lifecycle: __del__(). I don’t recommend customizing it — the official Python docs advise against it too — and I won’t cover it here. Instead of customizing __del__, just write a close() method.

Custom String Representations#

When we print an object we defined, the output looks like <__main__.Book object at 0x00000230AA7EA060>:

1
class Book:
2
    def __init__(self, title, author, isbn, stock):
3
        self.title = title
4
        self.author = author
5
        self.isbn = isbn
6
        self.stock = stock
7

8
fluent_python = Book("Fluent Python", "Luciano Ramalho", "978-1492056355", 3)
9
print(fluent_python)  # Output: <__main__.Book object at 0x00000230AA7EA060>

If you want an easy-to-read string representation of the object, it might seem like the only option is a get_book_info method — but is it really?

We can define an object’s string representation with the magic method __str__():

1
class Book:
2
    def __init__(self, title, author, isbn, stock):
3
        self.title = title
4
        self.author = author
5
        self.isbn = isbn
6
        self.stock = stock
7

8
    def __str__(self):
9
        return f'"{self.title}" by {self.author} (ISBN: {self.isbn}) - Stock: {self.stock}'
10

11
fluent_python = Book("Fluent Python", "Luciano Ramalho", "978-1492056355", 3)
12
print(fluent_python)  # Output: "Fluent Python" by Luciano Ramalho (ISBN: 978-1492056355) - Stock: 3

Much clearer now, isn’t it? But there’s a catch: from this output you can’t tell what attributes the class has or what values they hold — in short, it’s not formal enough. That’s when you define a __repr__():

1
class Book:
2
    def __init__(self, title, author, isbn, stock):
3
        self.title = title
4
        self.author = author
5
        self.isbn = isbn
6
        self.stock = stock
7

8
    def __str__(self):
9
        return f'"{self.title}" by {self.author} (ISBN: {self.isbn}) - Stock: {self.stock}'
10

11
    def __repr__(self):
12
        return f"Book(title={self.title}, author={self.author}, isbn={self.isbn}, stock={self.stock})"
13

14
fluent_python = Book("Fluent Python", "Luciano Ramalho", "978-1492056355", 3)
15
print(fluent_python)  # "Fluent Python" by Luciano Ramalho (ISBN: 978-1492056355) - Stock: 3
16
print(f"{fluent_python!s}")  # "Fluent Python" by Luciano Ramalho (ISBN: 978-1492056355) - Stock: 3
17
print(f"{fluent_python!r}")  # Book(title=Fluent Python, author=Luciano Ramalho, isbn=978-1492056355, stock=3)

When nothing is specified, printing shows what __str__ defines. When you explicitly ask for the formal representation, you get what __repr__ defines.

Comparisons with Magic Methods#

The magic methods used for comparisons are:

__eq__(self, other): defines equality behavior (==).
__ne__(self, other): defines inequality behavior (!=).
__lt__(self, other): defines less-than behavior (<).
__le__(self, other): defines less-than-or-equal behavior (<=).
__gt__(self, other): defines greater-than behavior (>).
__ge__(self, other): defines greater-than-or-equal behavior (>=).

Here’s an example: a class that represents a word. You might want to compare words in lexicographic (alphabetical) order — the default string comparison already does that — but you might also want to compare by other criteria, like word length or syllable count. In this example we’ll compare by length. Here’s the implementation:

1
class Word(str):
2

3
    def __new__(cls, word):
4
        # Note that we have to use __new__. This is because str is an immutable
5
        # type, so we have to initialize the value at creation time
6
        if ' ' in word:
7
            print("Value contains spaces. Truncating to first space.")
8
            word = word[:word.index(' ')]  # the word is all characters before the first space
9
        return str.__new__(cls, word)
10

11
    def __gt__(self, other):
12
        return len(self) > len(other)
13
    def __lt__(self, other):
14
        return len(self) < len(other)
15
    def __ge__(self, other):
16
        return len(self) >= len(other)
17
    def __le__(self, other):
18
        return len(self) <= len(other)
19

20
word1 = Word('hello')
21
word2 = Word('world')
22
word3 = Word('python')
23

24
print(word1 > word2)  # Output: False
25
print(word1 < word3)  # Output: True
26
print(word2 >= word1) # Output: True

Note that we didn’t define __eq__() and __ne__(). That’s deliberate: defining them would produce some strange results (for example, Word('foo') == Word('bar') would return True). So we leave them out and fall back on str’s built-in comparison.

Context Managers#

Context managers make sure resources are cleaned up properly after use. We can implement one with the magic methods __enter__() and __exit__():

1
class DatabaseConnection:
2
    def __init__(self, database_name):
3
        self.database_name = database_name
4
        self.connection = None
5

6
    def __enter__(self):
7
        print(f"Connecting to database: {self.database_name}")
8
        self.connection = self.database_name
9
        return self.connection
10

11
    def __exit__(self, exc_type, exc_value, traceback):
12
        print(f"Closing database connection: {self.database_name}")
13

14
with DatabaseConnection("user database") as conn:
15
    print(f"Using connection: {conn}")
16
    print("Performing database operations...")

with runs whatever __enter__() defines when entering the block and binds __enter__()’s return value to the target in the as clause (if there is one). When leaving the block, whatever __exit__() defines runs.

For async code, just switch to __aenter__() and __aexit__().

Optimizing Attribute Access with slots#

By default, Python objects store attributes in a dictionary. The upside is that you can add attributes dynamically; the downside is that it can use more memory and lookups are relatively slow. Defining __slots__ prevents the class from creating a __dict__ and a __weakref__.

To use __slots__, define an attribute named __slots__ in the class and give it the allowed attribute names:

1
class Book:
2
    __slots__ = ("title", "author", "isbn", "stock")
3

4
    def __init__(self, title, author, isbn, stock):
5
        self.title = title
6
        self.author = author
7
        self.isbn = isbn
8
        self.stock = stock

Note that without __weakref__, this class doesn’t support weak references. If you need weak reference support, declare __weakref__ in __slots__ manually.

With inheritance, if the parent class defines __slots__, the child class must define its own too — otherwise child instances will have a __dict__ and __weakref__.

Iterators and Generators#

You may have noticed that most container objects work with the for statement:

1
for char in "abc":
2
    print(char)

Behind the scenes, the for statement calls iter() on the container object. That function returns an iterator object with a __next__() method, which accesses the container’s elements one at a time. When there are no more elements, __next__() raises StopIteration to signal the loop to end. You can call __next__() via next(). Here’s the equivalent of for char in "abc" spelled out:

1
>>> s = 'abc'
2
>>> it = iter(s)
3
>>> next(it)
4
'a'
5
>>> next(it)
6
'b'
7
>>> next(it)
8
'c'
9
>>> next(it)
10
Traceback (most recent call last):
11
  File "<stdin>", line 1, in <module>
12
StopIteration

To make one of our own classes iterable, we just implement the above. Define __iter__() to return an object with a __next__() method; if the class already defines __next__(), just return self:

1
class Reverse:
2
    """Iterator for looping over a sequence backwards."""
3
    def __init__(self, data):
4
        self.data = data
5
        self.index = len(data)
6

7
    def __iter__(self):
8
        return self
9

10
    def __next__(self):
11
        if self.index == 0:
12
            raise StopIteration
13
        self.index = self.index - 1
14
        return self.data[self.index]

Now for a side note: is there an easier way to implement iteration? Yes — generators. Generators look like regular functions, but they use yield when they want to return data. They automatically create __iter__() and __next__(), and each call to next() on a generator resumes execution from where it left off. When all elements have been produced, it automatically raises StopIteration. With generators, creating an iterator can be as easy as writing a regular function:

1
def reverse(data):
2
    for index in range(len(data)-1, -1, -1):
3
        yield data[index]

With Metaclasses#

Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t. The people who actually need them know with certainty that they need them, and don’t need an explanation why.

——Tim Peters

So I decided not to cover them.

Just kidding — let’s continue.

Introduction#

Let’s start with “Hello, World!”:

1
print("Hello, World!".__class__)  # Output: <class 'str'>
2
print(str.__class__)  # Output: <class 'type'>
3
print(str.__bases__)  # Output: (<class 'object'>,)
4

5
print("Hello, World!".__bases__)  # Raise: AttributeError: 'str' object has no attribute '__bases__'. Did you mean: '__class__'?

In Python, everything is an object. Even basic data types are objects created by built-in classes (which is exactly why you can call methods directly on a string). As the code shows, the string "Hello, World!" is an instance of the built-in class str, and the string instance itself has no base classes. Turning to the built-in class str, it’s an instance of the type class, and its base class is object. Since instances are what you get when a class is instantiated, the built-in class str must itself be instantiated by type.

1
class Book:
2
    pass
3

4
print(Book.__class__)  # <class 'type'>
5
print(Book.__bases__)  # (<class 'object'>,)

The same holds for our custom classes: they inherit from object and are instances of type.

The Python Object System — From the Ground Up#

NOTE
This section is largely based on “Python Types and Objects” by Shalabh Chaturvedi, reorganized and adapted by me (Raven95676) for educational purposes. Since the original author retains all rights, this section is not released under the CC BY-NC-SA 4.0 license. If the original copyright holder believes this section infringes on their copyright, please contact me (Raven95676) to have the relevant content removed.
The images referenced in this section are quite old; please read any occurrence of <type 'object'> in them as <class 'object'>.

We’re going to build the Python object system from scratch, starting at the very beginning — a blank sheet of paper.

Before drawing anything, let’s introduce the relationships. We connect objects with two kinds of relationships:

Subclass-parent class relationship (inheritance): one object is a specialized version of another — “a catgirl is a kind of fantasy creature”.
Type-instance relationship (instantiation): one object is a concrete instance of another — “Nova the catgirl is an instance of catgirl”.

Inheritance is drawn as a solid line from child to parent; instantiation is a dashed line from instance to type, as in the diagram below:

Let’s start our exploration at the root of everything:

1
print(object.__class__)  # Output: <class 'type'>
2
print(object.__bases__)  # Output: ()
3

4
print(type.__class__)  # Output: <class 'type'>
5
print(type.__bases__)  # Output: (<class 'object'>,)

object is instantiated by type and has no parent class. That makes object the starting point of inheritance. type is instantiated by itself, and its parent is object.

Let’s put these findings on the diagram:

Now let’s bring in the built-in data types list, dict, and tuple:

1
print(list.__class__)  # Output: <class 'type'>
2
print(list.__bases__)  # Output: (<class 'object'>,)
3
print(tuple.__class__)  # Output: <class 'type'>
4
print(tuple.__bases__)  # Output: (<class 'object'>,)
5
print(dict.__class__)  # Output: <class 'type'>
6
print(dict.__bases__)  # Output: (<class 'object'>,)

They all inherit from object and are all instantiated by type. Now let’s instantiate a list and see what happens:

1
mylist = [1,2,3]
2
print(mylist.__class__)  # Output: <class 'list'>
3
print(mylist.__bases__)  # Raise: AttributeError: 'list' object has no attribute '__bases__'. Did you mean: '__class__'?

mylist is instantiated by list and has no parent class.

Let’s add these findings to the diagram:

Now let’s define our own class based on the existing object class (if you don’t specify a base, the class inherits from object by default), then instantiate it:

1
class C:
2
    pass
3

4
obj = object()
5
cobj = C()
6

7

8
print(C.__class__)  # Output: <class 'type'>
9
print(C.__bases__)  # Output: (<class 'object'>,)
10
print(obj.__class__)  # Output: <class 'object'>
11
print(cobj.__class__)  # Output: <class '__main__.C'>

Let’s put these findings on the diagram:

In fact, what each of these three boxes represents is now plain to see:

The first box is the class of all classes — we call it the metaclass.

The second box is both the class of the third box and an instance of the first box — we call it the type object.

The third box is an instance of the second box — we simply call it the instance.

The metaclass instantiates type objects, and type objects instantiate classes. In other words, the creation of every new object ultimately boils down to instantiation.

So the following two snippets are essentially equivalent:

1
class MyClass:
2
    pass
3

4
MyClass = type("MyClass", (), {})

Using Metaclasses#

So now we——

We know: by default, classes are built with type.

We know: classes can be inherited from.

So can we inherit from type, create a new metaclass, and use that new metaclass to create classes? Of course we can — that’s the very core of metaclasses.

Congratulations — we’re finally ready to learn how to use metaclasses.

A basic metaclass typically inherits from type and overrides its __new__() or __init__():

1
class Meta(type):
2
    def __new__(cls, clsname: str, bases: tuple, clsdict: dict):
3
        return super().__new__(cls, clsname, bases, clsdict)
4

5
    def __init__(self, clsname: str, bases: tuple, clsdict: dict):
6
        super().__init__(clsname, bases, clsdict)

Here, clsname is the class name and will become the new class’s __name__ attribute. bases is the tuple of base classes and will become the __bases__ attribute; if it’s empty, object is added. clsdict is the dict of attribute and method definitions from the class body; it may be copied or wrapped before becoming the __dict__ attribute.

Now let’s give this metaclass a job: it will reject any class definition containing method names with mixed case:

1
class NoMixedCaseMeta(type):
2
    def __new__(cls, clsname, bases, clsdict):
3
        for name in clsdict:
4
            if name.lower() != name:
5
                raise TypeError('Bad attribute name: ' + name)
6
        return super().__new__(cls, clsname, bases, clsdict)

How do you apply a metaclass? By specifying the metaclass parameter:

1
class A(metaclass=NoMixedCaseMeta):
2
    def foo_bar(self):
3
        pass
4

5
class B(metaclass=NoMixedCaseMeta):
6
    def fooBar(self):  # Raise: TypeError: Bad attribute name: fooBar
7
        pass

From our discussion above, we know classes are instantiated by a metaclass, which defaults to type. So specifying metaclass is equivalent to going from:

1
A = type('A', (), {'foo_bar': lambda self: None})
2
B = type('B', (), {'fooBar': lambda self: None})

to:

1
A = NoMixedCaseMeta('A', (), {'foo_bar': lambda self: None})
2
B = NoMixedCaseMeta('B', (), {'fooBar': lambda self: None})  # Raise: TypeError: Bad attribute name: fooBar

Practical OOP Tips in Python#

”Private” Attributes#

Python doesn’t have truly private attributes. But most Python code follows a convention: a name with a single leading underscore should be treated as a non-public part of the API (whether it’s a function, method, or data member). If that implementation ever disappears or its behavior changes, you’re on your own.

When a name has at least two leading underscores and at most one trailing underscore, it’s replaced with _classname__name, where classname is the current class name without the leading underscores. This mechanism is called name mangling. It’s generally used to avoid name clashes with names defined by subclasses.

Some tutorials online claim this mechanism can implement private attributes — that internal code can access the attribute while external code can’t. The second half of that claim is wrong: renaming a name doesn’t make it inaccessible. And personally, I wouldn’t use name mangling for this purpose — no matter how you mangle it, the attribute is still accessible from anywhere, so why bother?

Defining Interfaces with Abstract Base Classes#

In large multi-developer projects, how do you make sure classes written by different developers all follow a unified interface? That’s where abstract base classes come in. An ABC defines a set of methods that must be implemented; any subclass must implement every abstract method, or instantiating it raises an exception:

1
from abc import ABC, abstractmethod
2

3

4
class CacheInterface(ABC):
5
    @abstractmethod
6
    def save_to_cache(self, key, value):
7
        pass
8

9
    @abstractmethod
10
    def load_from_cache(self, key):
11
        pass
12

13

14
class RedisCache(CacheInterface):
15
    def save_to_cache(self, key, value):
16
        print(f"Saved to Redis: {key} = {value}")
17

18
    def load_from_cache(self, key):
19
        print(f"Loaded from Redis: {key}")
20
        return f"cached_{key}"
21

22

23
class FileCache(CacheInterface):
24
    def save_to_cache(self, key, value):
25
        print(f"Saved to file: {key} = {value}")
26

27
    def load_from_cache(self, key):
28
        print(f"Loaded from file: {key}")
29
        return f"file_{key}"
30

31

32
class IncompleteCache(CacheInterface):
33
    def save_to_cache(self, key, value):
34
        print(f"Saved: {key} = {value}")
35

36

37
redis_cache = RedisCache()
38
file_cache = FileCache()
39
test = IncompleteCache()  # Raise: TypeError: Can't instantiate abstract class IncompleteCache without an implementation for abstract method 'load_from_cache'

Using Enums#

When writing code we often need to represent a fixed set of states or types — order status (pending payment, paid, shipped), for example. Enums are a much more elegant way to do this:

1
from enum import Enum, auto
2

3
class OrderStatus(Enum):
4
    PENDING = "pending"
5
    PAID = "paid"
6
    SHIPPED = "shipped"
7
    DELIVERED = "delivered"
8
    CANCELLED = "cancelled"

If the concrete values don’t matter, use auto to generate them automatically:

1
class Priority(Enum):
2
    LOW = auto()
3
    MEDIUM = auto()
4
    HIGH = auto()

Eliminating Duplicate Code with Decorators#

Writing Your Own Decorators#

Generally speaking, a decorator is a function that takes another function as an argument and returns a closure with extra behavior added (hence the name “decorator”). Suppose we need functions that generate email content:

1
def welcome_message():
2
    print("Welcome to our team.")

Formal emails are expected to follow certain etiquette. But the added formatting and content are fixed — if every email-generating function copy-pasted the same boilerplate, a future change to the email format would force you to edit every function. To avoid that hassle, we can write a decorator:

1
def email_decorator(func):
2
    def wrapper():
3
        print("Hello,")
4
        func()
5
        print("Best regards,\nRaven")
6
    return wrapper

Let’s apply it:

1
@email_decorator
2
def welcome_message():
3
    print("Welcome to our team.")
4

5
welcome_message()

When we call welcome_message, what actually runs is the closure wrapper: it prints the greeting, calls the original function, and finally prints the signature.

In fact, the code above with @ is equivalent to this:

1
welcome_message = email_decorator(welcome_message)

@ is convenient syntactic sugar — decorators work perfectly fine without it.

Right now welcome_message takes no arguments. What if we want to extend it — say, to accept a department name? You might try this:

1
@email_decorator
2
def welcome_message(department):
3
    print(f"Welcome to our {department} team.")
4

5
welcome_message()  # Raise: TypeError: email_decorator.<locals>.wrapper() takes 0 positional arguments but 1 was given

But that raises a TypeError. Look closely: func() inside our decorator takes no arguments — that’s the root of the problem. To make the decorator more general, add *args and **kwargs so it accepts any number of arguments (though you can also add exactly the parameters you need):

1
def email_decorator(func):
2
    def wrapper(*args, **kwargs):
3
        print("Hello,")
4
        func(*args, **kwargs)
5
        print("Best regards,\nRaven")
6

7
    return wrapper

You may have noticed that this decorator returns nothing. But the decorated function may need to return a value — in that case the decorator has to “catch” that value and return it. Let’s use a decorator that measures a function’s execution time as an example:

1
import time
2

3
def timer_decorator(func):
4
    def wrapper(*args, **kwargs):
5
        start_time = time.time()
6
        result = func(*args, **kwargs)
7
        end_time = time.time()
8
        print(f"Function {func.__name__} took {end_time - start_time:.4f} seconds")
9
        return result
10
    return wrapper
11

12
@timer_decorator
13
def calc(n):
14
    def fibonacci(x):
15
        if x <= 1:
16
            return x
17
        return fibonacci(x - 1) + fibonacci(x - 2)
18

19
    result = fibonacci(n)
20

21
    return result
22

23
calc(35)

A detail many people miss: when you try to get metadata of the original function — like __name__ — you get wrapper instead of the intuitive calc. That’s expected, because wrapper is what’s actually being called. But sometimes you genuinely need the original function’s metadata. That’s where the wraps decorator from the functools module comes in:

1
import time
2

3
from functools import wraps
4

5

6
def timer_decorator(func):
7
    @wraps(func)
8
    def wrapper(*args, **kwargs):
9
        start_time = time.perf_counter()
10
        result = func(*args, **kwargs)
11
        end_time = time.perf_counter()
12
        print(f"Function {func.__name__} took {end_time - start_time:.4f} seconds")
13
        return result
14

15
    return wrapper

Back to our email decorator. Sometimes the decorator itself needs to take parameters — say, to customize the sender. In that case we add another layer on top of the existing decorator structure: a function called a decorator factory, which takes the parameters and “produces” a decorator:

1
from functools import wraps
2

3

4
def email_decorator(from_who):
5
    def decorator(func):
6
        @wraps(func)
7
        def wrapper(*args, **kwargs):
8
            print("Hello,")
9
            func(*args, **kwargs)
10
            print(f"Best regards,\n{from_who}")
11

12
        return wrapper
13

14
    return decorator
15

16

17
@email_decorator(from_who="Raven")
18
def welcome_message(department):
19
    print(f"Welcome to our {department} team.")
20

21

22
welcome_message("Development")

Why do we need a decorator factory? Can’t we just pass arguments directly? No — because the @ syntax only passes the decorated function to the decorator. If we want the decorator itself to accept arguments, passing them directly won’t work. The @ syntax above is equivalent to:

1
decorated_welcome = email_decorator(from_who="Raven")(welcome_message)

Here, email_decorator(from_who="Raven") runs first and returns a decorator function, which then receives welcome_message and produces the final decorated function.

Sometimes one decorator isn’t enough. In that case you simply stack them. One thing to keep in mind: decorators are applied from the bottom up, so don’t get the order wrong:

1
from functools import wraps
2

3

4
def email_decorator(from_who):
5
    def decorator(func):
6
        @wraps(func)
7
        def wrapper(*args, **kwargs):
8
            print("Hello,")
9
            func(*args, **kwargs)
10
            print(f"Best regards,\n{from_who}")
11

12
        return wrapper
13

14
    return decorator
15

16
def add_studio_info(func):
17
    @wraps(func)
18
    def wrapper(*args, **kwargs):
19
        result = func(*args, **kwargs)
20
        print("---")
21
        print("Raven-Studio")
22
        return result
23
    return wrapper
24

25
@add_studio_info
26
@email_decorator(from_who="Raven")
27
def welcome_message(department):
28
    print(f"Welcome to our {department} team.")
29

30

31
welcome_message("Development")

Actually, the order is easy to understand. The @ usage here is equivalent to:

1
welcome_message = add_studio_info(email_decorator(from_who="Raven")(welcome_message))

Besides functions, classes can also be used as decorators. This is very useful when you need to manage state or organize complex logic together, and it typically requires overriding two magic methods, __init__() and __call__():

1
from functools import wraps
2

3

4
class EmailDecorator:
5
    def __init__(self, from_who):
6
        self.from_who = from_who
7

8
    def __call__(self, func):
9
        @wraps(func)
10
        def wrapper(*args, **kwargs):
11
            print("Hello,")
12
            func(*args, **kwargs)
13
            print(f"Best regards,\n{self.from_who}")
14

15
        return wrapper
16

17

18
@EmailDecorator(from_who="Raven")
19
def welcome_message(department):
20
    print(f"Welcome to our {department} team.")
21

22

23
welcome_message("Development")

__init__() receives the decorator’s arguments; __call__() receives the function being decorated and returns the wrapped function.

Of course, decorators can also decorate classes — we’ll cover that together in the “Metaclass Alternatives” section below.

Using the lru_cache Decorator#

Now let’s turn back to the Fibonacci calculation function:

1
import time
2

3
from functools import wraps, lru_cache
4

5

6
def timer_decorator(func):
7
    @wraps(func)
8
    def wrapper(*args, **kwargs):
9
        start_time = time.perf_counter()
10
        result = func(*args, **kwargs)
11
        end_time = time.perf_counter()
12
        print(f"Function {func.__name__} took {end_time - start_time:.4f} seconds")
13
        return result
14
    return wrapper
15

16
@timer_decorator
17
def calc(n):
18
    @lru_cache(maxsize=4096)
19
    def fibonacci(x):
20
        if x <= 1:
21
            return x
22
        return fibonacci(x - 1) + fibonacci(x - 2)
23

24
    result = fibonacci(n)
25

26
    return result
27

28
calc(50)

@lru_cache caches the results of function calls. When the function is called again with the same arguments, it returns the cached result immediately instead of recomputing. Without @lru_cache, computing calc(50) takes a while; with it, the call finishes almost instantly.

Of course, @lru_cache isn’t suitable for every situation. First, the arguments must be hashable — the internal implementation relies on a dictionary that uses the arguments as keys. Second, the function must be called frequently with the same arguments for caching to pay off. And if the function depends on data that changes dynamically without being passed as arguments (like the current time or global state), a cached result may be stale.

Ensuring Unique Enum Values with @unique#

By default, enums allow multiple names to be aliases of the same value. If you need every value to be used only once, use the @unique decorator:

1
from enum import Enum, unique
2

3
@unique
4
class Mistake(Enum):
5
    ONE = 1
6
    TWO = 2
7
    THREE = 3
8
    FOUR = 3  # Raise: ValueError: duplicate values found in <enum 'Mistake'>: FOUR -> THREE

Using the property Decorator#

Suppose we’re developing some management system and need a Person class to manage a person’s information:

1
class Person:
2
    def __init__(self, name, age):
3
        self.name = name
4
        self.age = age
5

6
x = Person("M", 30)
7
print(x.age)  # Output: 30
8
x.age = 360
9
print(x.age)  # Output: 360

Here’s the problem: attribute assignment is completely uncontrolled — any value, no matter how unreasonable, gets accepted. We need validation when values are set. One option is writing set_xxx/get_xxx methods, but that isn’t very elegant in Python. Instead, use the @property decorator:

1
class Person:
2
    def __init__(self, name, age):
3
        self.name = name
4
        self._age = age
5

6
    @property
7
    def age(self):
8
        return self._age
9

10
    @age.setter
11
    def age(self, value):
12
        if 0 <= value < 150:
13
            self._age = value
14
        else:
15
            raise ValueError("Age must be between 0 and 150")
16

17
x = Person("M", 30)
18
print(x.age)  # Output: 30
19
x.age = 45
20
print(x.age)  # Output: 45
21
x.age = 360  # Raise: ValueError: Age must be between 0 and 150

Incidentally, if you don’t define a setter, you get a read-only attribute:

1
class Person:
2
    def __init__(self, name, age):
3
        self.name = name
4
        self._age = age
5

6
    @property
7
    def age(self):
8
        return self._age
9

10
x = Person("M", 30)
11
print(x.age)  # Output: 30
12
x.age = 360  # Raise: AttributeError: property 'age' of 'Person' object has no setter

You can also use deleter to define what happens when the attribute is deleted with del. For the record, if no deleter is defined, del raises an AttributeError:

1
class Person:
2
    def __init__(self, name, age):
3
        self.name = name
4
        self._age = age
5

6
    @property
7
    def age(self):
8
        return self._age
9

10
    @age.setter
11
    def age(self, value):
12
        if 0 <= value < 150:
13
            self._age = value
14
        else:
15
            raise ValueError("Age must be between 0 and 150")
16

17
    @age.deleter
18
    def age(self):
19
        self._age = 0
20

21
x = Person("M", 30)
22
print(x.age)  # Output: 30
23
del x.age
24
print(x.age)  # Output: 0

Using the dataclass Decorator#

When creating classes to store data in Python, you usually end up writing a lot of boilerplate. For example, a class to represent a product:

1
class Product:
2
    def __init__(self, name: str, unit_price: float, quantity: int = 0):
3
        self.name = name
4
        self.unit_price = unit_price
5
        self.quantity = quantity
6

7
    def __repr__(self):
8
        return f"Product(name='{self.name}', unit_price={self.unit_price}, quantity={self.quantity})"
9

10
    def __eq__(self, other):
11
        if not isinstance(other, Product):
12
            return NotImplemented
13
        return (self.name, self.unit_price, self.quantity) == (
14
            other.name,
15
            other.unit_price,
16
            other.quantity,
17
        )
18

19
    def total_cost(self) -> float:
20
        return self.unit_price * self.quantity

Every data class means reams of repetitive code — not very elegant. Fortunately, the @dataclass decorator generates all that boilerplate for us:

1
from dataclasses import dataclass
2

3
@dataclass
4
class Product:
5
    name: str
6
    unit_price: float
7
    quantity: int = 0
8

9
    def total_cost(self) -> float:
10
        return self.unit_price * self.quantity

When defining a dataclass, you can give fields default values. Note that all fields without defaults must come before fields with defaults, or you’ll get a TypeError. For immutable objects, assigning a default directly is fine. For mutable objects, you must use field’s default_factory parameter to avoid all instances sharing the same mutable object (after all, defaults live on the class as class attributes):

1
from dataclasses import dataclass, field
2

3

4
@dataclass
5
class Product:
6
    name: str
7
    unit_price: float
8
    quantity: int = 0
9
    tags: list[str] = field(default_factory=list)
10

11
    def total_cost(self) -> float:
12
        return self.unit_price * self.quantity

default_factory needs a callable that takes no arguments; the dataclass calls it to produce a fresh value whenever a default is needed. Passing list directly initializes an empty list. If you want the default to be a list with specific contents, use a lambda to define an anonymous function, e.g. lambda: ["content"].

By the way, if the default value is an immutable object, you can use the default parameter directly. Note that default and default_factory conflict — which makes sense: with two ways to initialize a default value, Python wouldn’t know which one to call.

And field can do more than that. If you don’t want a field to appear in __init__(), set init to False. If a field shouldn’t participate in comparisons, set compare to False. If a field should be keyword-only, set kw_only to True.

The dataclass decorator itself also has some very useful parameters. For example, setting frozen to True creates an immutable data class:

1
from dataclasses import dataclass
2

3
@dataclass(frozen=True)
4
class Point:
5
    x: int
6
    y: int
7

8
p = Point(1, 2)
9
p.x = 3  # Raise: dataclasses.FrozenInstanceError: cannot assign to field 'x'

You can also set order to True to make dataclass instances support ordering comparisons, not just equality. And if some value should be computed dynamically from other values after initialization, define a __post_init__ method:

1
import math
2

3
from dataclasses import dataclass, field
4

5
@dataclass
6
class Circle:
7
    radius: float
8
    area: float = field(init=False)
9

10
    def __post_init__(self):
11
        self.area = math.pi * (self.radius ** 2)
12

13
c = Circle(5.0)
14
print(c)  # Output: Circle(radius=5.0, area=78.53981633974483)

Metaclass Alternatives#

As I keep saying, 99% of the time you don’t need metaclasses. Lest anyone think that any kind of class customization requires a metaclass, here are the alternatives.

Class Decorators Instead of Metaclasses#

For one-off modifications or checks of a class, a class decorator works. Take the class-decorator equivalent of NoMixedCaseMeta as an example:

1
def no_mixed_case(cls):
2
    for name in cls.__dict__:
3
        if name.lower() != name:
4
            raise TypeError('Bad attribute name: ' + name)
5

6
    return cls
7

8

9
@no_mixed_case
10
class A:
11
    def foo_bar(self):
12
        pass
13

14
@no_mixed_case
15
class B:
16
    def fooBar(self):  # Raise: TypeError: Bad attribute name: fooBar
17
        pass

Magic Methods Instead of Metaclasses#

What if you need the check to happen at inheritance time? That’s where the __init_subclass__ magic method comes in. Again using the equivalent of NoMixedCaseMeta as an example:

1
class NoMixedCaseBase:
2
    def __init_subclass__(cls, *args, **kwargs):
3
        super().__init_subclass__(*args, **kwargs)
4

5
        for name in cls.__dict__:
6
            if name.lower() != name:
7
                raise TypeError("Bad attribute name: " + name)
8

9

10
class A(NoMixedCaseBase):
11
    def foo_bar(self):
12
        pass
13

14

15
class B(NoMixedCaseBase):
16
    def fooBar(self):  # Raise: TypeError: Bad attribute name: fooBar
17
        pass

When Only a Metaclass Will Do#

Is there anything that truly requires a metaclass — something only a metaclass can achieve? Yes. For instance, suppose we get the wild idea to reverse the MRO. That calls for a metaclass:

1
class PluginA:
2
    def test(self):
3
        return "Plugin A"
4

5

6
class PluginB:
7
    def test(self):
8
        return "Plugin B"
9

10

11
class ReverseMeta(type):
12
    def mro(cls):
13
        default_mro = list(type.mro(cls))
14
        reversed_mro = list(reversed(default_mro[:-1])) + [object]
15
        return tuple(reversed_mro)
16

17

18
class A(PluginA, PluginB):
19
    pass
20

21
class B(PluginA, PluginB, metaclass=ReverseMeta):
22
    pass
23

24
print(A.__mro__)  # (<class '__main__.A'>, <class '__main__.PluginA'>, <class '__main__.PluginB'>, <class 'object'>)
25
print(B.__mro__)  # (<class '__main__.PluginB'>, <class '__main__.PluginA'>, <class '__main__.B'>, <class 'object'>)
26

27
a = A()
28
b = B()
29

30
print(a.test())  # Output: Plugin A
31
print(b.test())  # Output: Plugin B

OK, let’s look at a more down-to-earth example: implementing a minimal ORM.

1
class Field:
2
    def __init__(self, default=None):
3
        self.default = default
4

5
class CharField(Field):
6
    def __init__(self, default=""):
7
        super().__init__(default)
8

9
class IntegerField(Field):
10
    def __init__(self, default=0):
11
        super().__init__(default)
12

13
class ModelMeta(type):
14
    def __new__(mcs, name, bases, clsdict):
15
        if name == 'Model':
16
            return super().__new__(mcs, name, bases, clsdict)
17
        fields = {}
18
        for k, v in clsdict.items():
19
            if isinstance(v, Field):
20
                fields[k] = v
21
        for k in fields:
22
            del clsdict[k]
23
        clsdict['__fields__'] = fields
24
        return super().__new__(mcs, name, bases, clsdict)
25

26
class Model(metaclass=ModelMeta):
27
    def __init__(self, **kwargs):
28
        for field_name, field in self.__fields__.items():
29
            value = kwargs.get(field_name, field.default)
30
            setattr(self, field_name, value)
31

32
    def __repr__(self):
33
        field_values = []
34
        for field_name in self.__fields__:
35
            value = getattr(self, field_name)
36
            field_values.append(f"{field_name}={value!r}")
37
        return f"{self.__class__.__name__}({', '.join(field_values)})"
38

39
class Product(Model):
40
    title = CharField()
41
    price = IntegerField()
42
    description = CharField(default="No description")
43

44
product1 = Product(title="Laptop", price=7999, description="Portable computer")
45
product2 = Product(title="Phone", price=4999)
46

47
print(product1)  # Output: Product(title='Laptop', price=7999, description='Portable computer')
48
print(product2)  # Output: Product(title='Phone', price=4999, description='No description')

OOP Best Practices#

Code is written for humans to read. Unless you have a compelling reason to do otherwise — in which case leave a comment explaining it — or the code is throwaway, don’t force your future self, or the colleague inheriting your code, to become an archaeologist.

——Raven

KISS#

Keep it simple stupid — this is the overarching principle of best practices. Always pursue simple, clear design and implementation, and avoid unnecessary complexity.

Here’s the simplest example. Suppose we need a function that takes a list of integers and returns the squares of all even numbers in it. A more convoluted version looks like this:

1
def process_data(data: list[int]) -> list[int]:
2
    result = []
3
    temp = []
4
    for item in data:
5
        if item % 2 == 0:
6
            temp.append(item)
7
    for num in temp:
8
        result.append(num**2)
9
    return result

It only takes a single list comprehension:

1
def process_data(data: list[int]) -> list[int]:
2
    return [x**2 for x in data if x % 2 == 0]

YAGNI#

YAGNI stands for “You aren’t gonna need it”. When writing code, don’t overthink it — get the task in front of you done first, and adapt when the need actually arises. Don’t assume “what if we need it someday?”. Unless there’s a concrete requirement, that “someday” is just a possibility — not worth pouring a lot of effort into.

SOLID#

NOTE
This section draws on the relevant definitions from Wikipedia. Since Wikipedia content is licensed under CC BY-SA 4.0, this section is not released under the CC BY-NC-SA 4.0 license.
This section is largely based on “Have you really understood SOLID after all these years of coding?” by Mei Xuesong.

(Image source: “Have you really understood SOLID after all these years of coding?” by Mei Xuesong)

In short, the Single Responsibility Principle is the foundation of all design principles, and the Open/Closed Principle is the ultimate goal of design. The Liskov Substitution Principle emphasizes the correctness of a program at runtime when subclasses replace their parent classes, and it helps achieve the Open/Closed Principle. The Interface Segregation Principle helps achieve the Liskov Substitution Principle while also embodying the Single Responsibility Principle. The Dependency Inversion Principle is the dividing line between procedural programming and object-oriented programming, and it also guides the Interface Segregation Principle.

——Mei Xuesong

S - Single Responsibility Principle#

An object should have only a single responsibility.

Per Robert C. Martin’s definition, the core idea of the Single Responsibility Principle is to use “reasons to change” as the criterion for defining responsibilities: every class or module should have exactly one reason to change. If you can think of more than one reason to change a class, that class has more than one responsibility.

Take a report processing module as an example. Imagine a module that both edits and prints reports. Such a module has two reasons to change. First, the content of the report can change (editing). Second, the format of the report can change (printing). These two kinds of changes happen for entirely different reasons. The Single Responsibility Principle holds that these are in fact two separate responsibilities, and so they should live in separate classes or modules.

Coupling things that change for different reasons is bad design. If the report editing flow changes, there’s serious danger: when both responsibilities live in the same class, a change to the editing flow can alter shared state or dependencies and break the printing code.

Frequency of change is another reason worth considering. Even within the same role, requirements change at different rates. The classic case: business logic requirements are relatively stable, while presentation requirements change much more often — people always want something new. So these two kinds of requirements are typically implemented in different classes.

In a sense, the Single Responsibility Principle is about separating concerns: separating the concerns of different roles, and separating concerns that change at different times.

How do you apply the Single Responsibility Principle in practice? When do you split, and when do you merge? Watch a new chef learning to stir-fry and figuring out “a pinch of salt”. He keeps tasting until the flavor is just right. Writing code works the same way: you need to recognize the signals of changing requirements, keep “tasting” your code, and refactor until the “flavor” is exactly right.

O - Open/Closed Principle#

Software should be open for extension, but closed for modification.

A well-designed entity should be able to change its behavior without modifying its source code.

Why? Suppose you’re the author of a successful open-source library used by many developers. One day you need to extend a feature, and the only way is to modify existing code — which forces every library user to change their code too. Worse, after they’re forced to make those changes, their own dependents may end up forced to change theirs as well. That scenario is an absolute disaster.

That said, we can’t foresee every possible extension point at initial design time, and we can’t reserve extension points everywhere. The right approach is to let requirement changes drive design decisions (which is YAGNI).

We may never fully achieve the Open/Closed Principle, but that doesn’t stop it from being the ultimate goal of design. Every other SOLID principle serves the Open/Closed Principle, directly or indirectly.

L - Liskov Substitution Principle#

Objects in a program should be replaceable by instances of their subtypes without altering the correctness of that program.

Anywhere a parent class is used in a program, a subclass should be substitutable without changing the program’s behavior. A subclass should extend the parent’s behavior, not alter it. The classic example comes from Robert C. Martin’s “Agile Software Development: Principles, Patterns, and Practices”: the square inheriting from the rectangle.

1
class Rectangle:
2
    def __init__(self, width: int, height: int) -> None:
3
        self._width = width
4
        self._height = height
5

6
    @property
7
    def width(self) -> int:
8
        return self._width
9

10
    @width.setter
11
    def width(self, value: int):
12
        self._width = value
13

14
    @property
15
    def height(self) -> int:
16
        return self._height
17

18
    @height.setter
19
    def height(self, value: int):
20
        self._height = value
21

22
    @property
23
    def area(self) -> int:
24
        return self._width * self._height
25

26

27
class Square(Rectangle):
28
    def __init__(self, size: int):
29
        super().__init__(size, size)
30

31
    @property
32
    def width(self) -> int:
33
        return super().width
34

35
    @width.setter
36
    def width(self, value: int):
37
        self._width = value
38
        self._height = value
39

40
    @property
41
    def height(self) -> int:
42
        return super().height
43

44
    @height.setter
45
    def height(self, value: int):
46
        self._width = value
47
        self._height = value
48

49

50
def print_area(rect: Rectangle):
51
    rect.width = 5
52
    rect.height = 10
53
    print(f"Area: {rect.area}")
54

55

56
rect = Rectangle(2, 3)
57
print_area(rect)  # Output: Area: 50
58

59
sq = Square(5)
60
print_area(sq)  # Output: Area: 100  <-- behavior has changed

Common sense says a square is a special kind of rectangle, so square-inherits-from-rectangle seems harmless. But print_area assumes that after setting width to 5 and height to 10, the passed-in Rectangle has an area of 50. When a Square is passed in, Square enforces equal width and height, so setting height to 10 also sets width to 10, making the area 100. Thus Square cannot fully replace Rectangle: it changes the behavior print_area expects.

In short, the conflict stems from the behavioral contract implied by Rectangle’s methods. Rectangle’s setters imply that “width and height can be modified independently”, and Square’s implementation breaks that contract — which is why it violates the Liskov Substitution Principle.

If your design satisfies the Liskov Substitution Principle, subclasses (or implementations of an interface) can replace their parents (or the interface) without breaking correctness, changing the system’s behavior and thereby extending it. Both Branch By Abstraction (introducing an abstraction layer that the old and new implementations both conform to, then gradually replacing functionality) and the Strangler pattern (during the transition period when old and new systems coexist, routing requests to old and new implementations through an abstraction layer, gradually migrating functionality to the new system) build on the Liskov Substitution Principle to extend and evolve systems. That is precisely “closed for modification, open for extension” — which makes the Liskov Substitution Principle a way to achieve the Open/Closed Principle.

And to achieve the Liskov Substitution Principle, you need the Interface Segregation Principle.

I - Interface Segregation Principle#

Many client-specific interfaces are better than one general-purpose interface.

Interfaces exist to decouple. Developers often have a misconception that implementation classes need interfaces. In fact, consumers need interfaces; implementation classes merely provide services. So interfaces should be defined by the consumer (the client). Only with this understanding can you correctly stand in the consumer’s shoes and define Role interfaces, instead of extracting Header interfaces from implementation classes.

For example, say we have a modern printer with print, scan, and fax capabilities. Extracting an interface directly from it like this produces a Header interface:

1
from abc import ABC, abstractmethod
2

3

4
class PrinterInterface(ABC):
5
    @abstractmethod
6
    def print_doc(self, doc: str):
7
        pass
8

9
    @abstractmethod
10
    def scan_doc(self):
11
        pass
12

13
    @abstractmethod
14
    def fax_doc(self, doc: str, num: str):
15
        pass

But here’s the problem: suppose we have an old printer that can only print — no scanning or faxing. The old printer is forced to depend on interface methods it doesn’t need, which violates the Interface Segregation Principle:

1
class OldPrinter(PrinterInterface):
2
    def print_doc(self, doc: str):
3
        print(f"Printing document: {doc}")
4

5
    def scan_doc(self):
6
        raise NotImplementedError("This printer does not support scanning")
7

8
    def fax_doc(self, doc: str, num: str):
9
        raise NotImplementedError("This printer does not support faxing")

We can’t guarantee that clients depending on this interface will never call the unsupported methods. So the right approach is to stand in the consumer’s shoes and abstract out Role interfaces:

1
from abc import ABC, abstractmethod
2

3

4
class Printable(ABC):
5
    @abstractmethod
6
    def print_doc(self, doc: str):
7
        pass
8

9

10
class Scannable(ABC):
11
    @abstractmethod
12
    def scan_doc(self):
13
        pass
14

15

16
class Faxable(ABC):
17
    @abstractmethod
18
    def fax_doc(self, doc: str, num: str):
19
        pass
20

21

22
class OldPrinter(Printable):
23
    def print_doc(self, doc: str):
24
        print(f"Printing document: {doc}")
25

26

27
class ModernPrinter(Printable, Scannable, Faxable):
28
    def print_doc(self, doc: str):
29
        print(f"Printing document: {doc}")
30

31
    def scan_doc(self):
32
        print("Scanning document")
33

34
    def fax_doc(self, doc: str, num: str):
35
        print(f"Sending fax: {doc} to {num}")

With Role interfaces, the old printer and the modern printer — each a consumer — can consume exactly the interfaces they need.

In essence, the Interface Segregation Principle is itself an expression of the Single Responsibility Principle, and it serves the Liskov Substitution Principle in turn.

D - Dependency Inversion Principle#

Depend upon abstractions, not concretions.

This principle is really guidance on how to implement the Interface Segregation Principle, as mentioned earlier: high-level consumers should not depend on concrete implementations; consumers should define and depend on Role interfaces, and low-level concrete implementations should also depend on Role interfaces, since they implement those interfaces.

(Image source: A Simple DIP Example)

In the diagram above, when the Button directly switches the lamp on and off, the Button depends on the Lamp. That code is completely equivalent to procedural programming:

1
class Lamp:
2
    def turn_on(self):
3
        print("Lamp is ON")
4

5
    def turn_off(self):
6
        print("Lamp is OFF")
7

8

9
class Button:
10
    def __init__(self, lamp: Lamp):
11
        self.lamp = lamp
12

13
    def press(self):
14
        self.lamp.turn_on()
15

16
    def release(self):
17
        self.lamp.turn_off()
18

19

20
lamp = Lamp()
21
button = Button(lamp)
22
button.press()  # Output: Lamp is ON
23
button.release()  # Output: Lamp is OFF

(Image source: A Simple DIP Example)

What if the Button also needs to control a TV, or a microwave? The way to handle this kind of change is abstraction: abstract out a Role interface, ButtonServer:

1
from abc import ABC, abstractmethod
2

3

4
class ButtonServer(ABC):
5
    @abstractmethod
6
    def turn_on(self):
7
        pass
8

9
    @abstractmethod
10
    def turn_off(self):
11
        pass
12

13

14
class Lamp(ButtonServer):
15
    def turn_on(self):
16
        print("Lamp is ON")
17

18
    def turn_off(self):
19
        print("Lamp is OFF")
20

21

22
class TV(ButtonServer):
23
    def turn_on(self):
24
        print("TV is ON")
25

26
    def turn_off(self):
27
        print("TV is OFF")
28

29

30
class Button:
31
    def __init__(self, device: ButtonServer):
32
        self.device = device
33

34
    def press(self):
35
        self.device.turn_on()
36

37
    def release(self):
38
        self.device.turn_off()
39

40

41
lamp = Lamp()
42
tv = TV()
43

44
button1 = Button(lamp)
45
button2 = Button(tv)
46

47
button1.press()  # Output: Lamp is ON
48
button1.release()  # Output: Lamp is OFF
49
button2.press()  # Output: TV is ON
50
button2.release()  # Output: TV is OFF

Whether it’s a lamp or a TV, as long as it implements ButtonServer, the Button can control it. This is the object-oriented way of programming.

DRY#

DRY stands for “Don’t repeat yourself”. When code is highly repetitive (for example, features built through lots of copy-paste), it’s time to think hard about whether there’s a better solution. In Python, decorators are usually the go-to tool for this kind of problem.

Summary#

Every best practice above boils down to two operations: “decouple” and “reuse”.

KISS emphasizes simplicity, and one of the main ways to achieve simplicity is to eliminate unnecessary complex dependencies through sensible decoupling, and to avoid repeated implementation through effective reuse. YAGNI reminds us that decoupling and reuse should be driven by actual requirements rather than speculation — don’t decouple for the sake of decoupling, or reuse for the sake of reuse. SOLID and DRY provide concrete guidelines for decoupling and reuse. And decoupling and reuse are themselves related: reuse requires decoupling — only when modules are fully decoupled from each other can they be effectively reused.

All these principles point to the same goal: building software systems that are maintainable, extensible, robust, and easy to understand.

References#

[Latest] The Python Tutorial
[Latest] The Python Language Reference
[2015] Python Crash Course, 3rd Edition
[2011] Python Cookbook, 3rd Edition
[Unknown] Runoob Python 3 Tutorial
[2012] A Guide to Python’s Magic Methods
[2006] A Simple DIP Example
[2018] Have you really understood SOLID after all these years of coding? By Mei Xuesong (URL no longer reachable): hxxps://insights.thoughtworks.cn/what-is-solid-principle/
[2009] Python Types and Objects By Shalabh Chaturvedi (URL no longer reachable): hxxp://www.cafepy.com/article/python_types_and_objects/python_types_and_objects.html

Useful Registry Keys (Continuously Updated)

Thu, 23 Jan 2025 00:00:00 GMT

WARNING
Editing the registry carries risks — proceed with caution. Incorrect changes can cause serious system failures, or even require a full system reinstall.

NOTE
Thanks to Disa for the additions.

Auto-start#

Run: runs at every boot.
RunOnce: runs only once, then deletes itself after execution.

User-level#

1
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
2
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnce
3
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnce\Setup
4
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunServices
5
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunServicesOnce
6
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer\Run
7
HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows\load

System-level#

1
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
2
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
3
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce\Setup
4
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnceEx
5
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunServices
6
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunServicesOnce
7
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\Explorer\Run
8
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\Userinit
9
HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Run
10
HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\RunOnce

Image File Execution Options#

Image hijacking#

When you launch the target program, the hijacking program runs instead of the original one.

1
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Option

Create a subkey named after the target process, and inside it add a debugger value set to the hijacking program.

SilentProcessExit#

When ReportingMode is set to 1, the system runs the program specified in MonitorProcess when the monitored process exits.

1
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SilentProcessExit

Create a subkey named after the target process, add a MonitorProcess value set to the hijacking program, and a ReportingMode value set to 1.

MinimumStackCommitInBytes#

Setting MinimumStackCommitInBytes to a large enough value makes the specified program crash with a stack overflow.

1
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\[Target Process]\MinimumStackCommitInBytes

Scheduled Tasks#

Hierarchy information#

1
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tree

If the index is 0, the task is not visible in taskschd.msc.

Task details#

1
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tasks

Deleting Files After Reboot#

1
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations

Each operation consists of a pair of strings:

The first string is the path of the source file or directory.
The second string is the path of the target file or directory (empty for a delete operation).

For example:

1
\??\C:\OldFile.txt
2
\??\C:\NewFile.txt

This renames C:\OldFile.txt to C:\NewFile.txt.

Unlocking BitLocker from the Command Line

Fri, 20 Dec 2024 00:00:00 GMT

The commands below assume the drive to be unlocked is C, unlocked using the 48-digit recovery key.

CMD:

1
 manage-bde.exe -unlock C: -recoverypassword xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx

PowerShell:

1
 Unlock-BitLocker -MountPoint C -RecoveryPassword xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx

Basic Security Measures for a VPS

Fri, 22 Nov 2024 00:00:00 GMT

This article is a self-reposted copy at: https://linux.do/t/topic/267502

Preface#

When you get your hands on a VPS, what should you do with it?

If you host your website or services on shared hosting from a reputable provider, the provider handles the basic server security. On a VPS, though, you are the one responsible for security. More privileges come with more responsibility — a VPS gives you far more freedom and, naturally, carries far more risk. The goal isn’t absolute security; it’s being more secure than most people. If your server isn’t easy to break into, that’s a win.

This article is written for Ubuntu 24.04 LTS; the general ideas still apply to other distributions.

Before you continue, we recommend reading the material by LUG@USTC: https://101.lug.ustc.edu.cn/

NOTE
This article is unrelated to the https://github.com/ustclug/Linux101-docs project.

Account Security#

Some VPS providers hand you the root account by default. Root holds the highest privileges on the entire system, and leaving that much power exposed by default is naturally risky. The right approach is to create a non-root account and use sudo to elevate privileges only when root access is truly needed.

Create a Non-root Account#

Create an account that can elevate privileges with the following command:

1
useradd -m -G sudo -s /bin/bash <username>

Then set a password for this user — at least 16 characters of mixed upper/lowercase letters and digits (my personal minimum for acceptable security):

1
passwd <username>

You should also change the root password. The default passwords on cloud servers are usually weak, and some providers even email the default credentials, which isn’t great for security.

Leaving aside the fact that logging in as root is inherently risky, the root username is always “root” — if password login is allowed for it, an attacker only needs to brute-force the password to try to break into your system. Since we’ve already created a non-root account, we can simply disable SSH login for root.

Edit the SSH configuration file:

1
sudo vim /etc/ssh/sshd_config

Set the following:

1
# Disable remote password login for the root user
2
PermitRootLogin prohibit-password

Then restart the SSH service to apply the change:

1
sudo systemctl restart ssh

Why not set it to no?

I don’t usually fully disable root login with PermitRootLogin no either. I’m more used to setting up keys like any other sudoer and disabling password login entirely, allowing keys only. I only configure it that way on machines where I can operate the out-of-band cluster console; otherwise one failure could leave you without root — for example, if the disk fills up, you can’t even establish a remote SSH session.

In theory PermitRootLogin prohibit-password is fine, but it becomes a hassle when the server dies and you need rescue access. I’d actually recommend only disabling remote password login while keeping local password login, so you can still log in as root locally via VNC or IPMI to rescue the machine when something goes wrong.

Change the SSH Port#

The very first thing is to move SSH off port 22.

This is a supplement added on top of the original post.

In practice, most people change the port as soon as they log in.

The first thing I do after logging in is change the sshd port and update the firewall — port 22 gets nothing but password-guessing attempts.

One trick: after changing the port and restarting sshd, your current connection stays alive. Try connecting to the new port — if it connects, the change worked. If you can’t establish a new connection, you can still revert the change or check your firewall configuration.

Normally, editing the SSH port via sudo vim /etc/ssh/sshd_config and restarting with sudo systemctl restart ssh.service is enough:

1
# Set the SSH port
2
Port <your-port>

However, on Ubuntu 22.10 or newer you may find this is ineffective — after restarting, the SSH service still listens on the original port.

That’s because on Ubuntu 22.10 and newer, ssh is socket-activated by default.

On Ubuntu 22.10, Ubuntu 23.04, and Ubuntu 23.10, change it like this:

1
sudo mkdir -p /etc/systemd/system/ssh.socket.d
2
sudo vim /etc/systemd/system/ssh.socket.d/listen.conf
3
sudo systemctl daemon-reload
4
sudo systemctl restart ssh.socket
5
sudo systemctl restart ssh.service

A reference listen.conf:

1
[Socket]
2
ListenStream=
3
ListenStream=2233

On Ubuntu 24.04, change it like this:

1
sudo vim /etc/ssh/sshd_config
2
sudo systemctl daemon-reload
3
sudo systemctl restart ssh.service

If you don’t care about the memory saved by socket activation, you can revert to a non-socket-activated setup with the following commands:

WARNING
Make sure your configuration files are correct first.

1
sudo systemctl disable --now ssh.socket
2
sudo systemctl enable --now ssh.service

If you’re migrating a configuration (from Ubuntu 22.10 and above, below 24.04):

1
sudo systemctl disable --now ssh.socket
2
rm -f /etc/systemd/system/ssh.service.d/00-socket.conf
3
rm -f /etc/systemd/system/ssh.socket.d/addresses.conf
4
sudo systemctl daemon-reload
5
sudo systemctl enable --now ssh.service

Instead of editing /etc/ssh/sshd_config directly, it’s better to create a conf file under /etc/ssh/sshd_config.d/ for your custom sshd settings. This prevents configuration conflicts after an OpenSSH update.

Reference: ssh - PasswordAuthentication no, but I can still login by password - Unix & Linux Stack Exchange

Some cloud providers drop their own conf file into /etc/ssh/sshd_config.d/ to enable remote password login (sshd disables it by default). Rule those out before modifying any sshd settings:

1
# Check whether sshd_config.d contains other conf files
2
sudo ls /etc/ssh/sshd_config.d/*.conf
3
# If so, rename them so your custom config isn't overridden
4
sudo mv /etc/ssh/sshd_config.d/xxx.conf/etc/ssh/sshd_config.d/xxx.conf.bak

CloudCone, for example, ships a /etc/ssh/sshd_config.d/50-cloud-init.conf.

After changing sshd settings, run sudo sshd -T first to inspect the effective configuration, so you’re not blindsided by an override :joy:.

1
# How root is allowed to log in
2
sudo sshd -T | grep -i "PermitRootLogin"
3
# Password authentication
4
sudo sshd -T | grep -i "PasswordAuthentication"
5
# SSH port
6
sudo sshd -T | grep -i "Port"

Worth noting: prohibit-password is an alias of without-password, so seeing the output below is perfectly normal.

1
$ sudo sshd -T | grep -i "PermitRootLogin"
2
permitrootlogin without-password

References:

Protect SSH from Brute Force with Fail2ban#

Install Fail2ban:

1
sudo apt install fail2ban

The officially recommended approach is to customize settings via jail.local:

1
sudo vim /etc/fail2ban/jail.local

Use the following config as a starting point for your own (remember to remove the comments):

1
[sshd]
2
ignoreip = 127.0.0.1/8 # Whitelist
3
enabled = true
4
filter = sshd
5
port = 22 # Port; update this if you changed the SSH port
6
maxretry = 5 # Maximum number of attempts
7
findtime = 300 # Window in seconds within which maxretry applies
8
bantime = 600 # Ban duration in seconds; -1 is a permanent ban (not recommended)
9
action = %(action_) s [port="%(port) s", protocol="%(protocol) s", logpath="%(logpath) s", chain="%(chain) s"] # Use this if you don't need email notifications
10
banaction = iptables-multiport # Ban method
11
logpath = /var/log/auth.log # SSH login log location

Very useful. I also added a WeCom bot notification that fires automatically on every SSH login, so I’d know right away if someone got in.

You can use a PAM module to trigger a script on every SSH login.

Edit /etc/pam.d/sshd and append:

1
session    optional    pam_exec.so /path/to/script

For the use case above, the script looks roughly like this: Reference: https://developer.work.weixin.qq.com/document/path/91770

Note (Nov 26, 2024): Fixed some issues in the example script.

1
#!/bin/bash
2

3
if [ "$PAM_TYPE" != "open_session" ]; then
4
    exit 0
5
fi
6

7
ip=$PAM_RHOST
8
date=$(date +"% e % b % Y, % a % r")
9
name=$PAM_USER
10

11
webhook_url="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxxxxxxxxxxxxxx"
12

13
curl -s -X POST "$webhook_url" \
14
    -H "Content-Type: application/json" \
15
    -d "{
16
    \"msgtype\": \"markdown\",
17
    \"markdown\": {
18
        \"content\": \"**Login Alert**\\n> User: $name\\n> Client IP: $ip\\n> Login time: $date\"
19
    }
20
}"

Replace the API key and delivery method with whatever fits your own use case.

Once the script is written, make it executable:

sudo chmod +x /usr/local/bin/notify_ssh_login.sh

[Strict] Run Commands with Absolute Paths#

Using absolute paths pins down exactly which program or file runs, so a tampered environment variable can’t trick you into executing a potentially malicious program.

Using the full path to su avoids running a fake su planted by an attacker.

Log In with SSH Keys#

TIP
If your VPS provider offers SSH key binding, you can skip this section and follow the provider’s method.

Run this in PowerShell:

1
ssh-keygen -t ed25519

Just accept the default key path. You can leave the passphrase empty or set one.

1
Generating public/private ed25519 key pair.
2
Enter file in which to save the key (C:\Users\<user>/.ssh/id_ed25519): # Press Enter to accept the default
3
Enter passphrase (empty for no passphrase): # Leave empty or set a passphrase
4
Enter same passphrase again: # Same as above

Then edit the authorized keys file on the VPS:

1
vim ~/.ssh/authorized_keys

Next, open C:\Users<user>/.ssh/id_ed25519.pub, copy its contents, and paste them in.

Edit the SSH configuration file:

1
sudo vim /etc/ssh/sshd_config

Set the following:

1
PubkeyAuthentication yes
2
AuthorizedKeysFile .ssh/authorized_keys
3
PasswordAuthentication no

Then restart the SSH service to apply the change:

1
sudo systemctl restart ssh

Enable the UFW Firewall#

TIP
If your VPS provider offers a firewall and you don’t have complex requirements, you can skip this section and use the provider’s firewall.

Before enabling UFW, we need to set up rules first. Let’s start with UFW’s default policies:

1
sudo ufw default allow outgoing # Allow all outbound traffic by default
2
sudo ufw default deny incoming # Deny all inbound traffic by default

View the currently active UFW rules:

1
sudo ufw status
2
sudo ufw status numbered  # Show rules with numbers

Allow or deny traffic on a port with the following commands (22, 80, and 443 used as examples):

1
# Allow inbound traffic on port 22/proto
2
sudo ufw allow in 22/proto
3

4
# Allow outbound traffic on port 22/proto
5
sudo ufw allow out 22/proto
6

7
# Without specifying in/out, it defaults to in
8
sudo ufw allow 22/proto
9

10
# Without specifying a protocol, both tcp and udp are allowed
11
sudo ufw allow 22
12

13
# To deny instead, change allow to deny
14
sudo ufw deny 22
15

16
# Allow a port range from start_port to end_port
17
sudo ufw allow start_port:end_port
18

19
# Allow multiple ports, comma-separated
20
sudo ufw allow port1,port2
21

22
# Allow traffic from a specific IP or CIDR range
23
sudo ufw allow from ip/cidr
24

25
# Allow traffic from a specific IP or CIDR range to port 22
26
sudo ufw allow from ip/cidr to any port 22
27

28
# Allow tcp traffic from a specific IP or CIDR range to port 22
29
sudo ufw allow from ip/cidr to any proto tcp port 22
30

31
# If you specify multiple ports, you must specify the protocol
32
sudo ufw allow from ip to any proto tcp port 80,443
33

34
# comment adds a description to the rule
35
sudo ufw allow from ip to any proto tcp port 80,443 comment "hello"

Delete active rules with:

1
sudo ufw delete allow 22 # Prefix the rule with delete
2
sudo ufw delete 1 # Or delete by the number from "ufw status numbered"

Once all rules are in place, start, stop, or reload UFW:

WARNING
Before enabling the firewall, make sure port 22 (or whatever your SSH port is) is allowed.

1
sudo ufw enable|disable|reload

To reset all rules:

WARNING
Make sure UFW is disabled before resetting the rules.

1
sudo ufw reset

My advice: only allow the ports you actually use, such as 22, 80, and 443.

By default, UFW only logs dropped packets that don’t match a rule. If you want detailed logging for a service, add log after allow:

1
# Log successful SSH connections too, for the record
2
ufw allow log 22/tcp

Block Ping#

Block Ping with iptables#

Check current iptables rules:
- iptables -L -n
- This lists the current iptables rules: -L lists the rules, and -n displays IP addresses and port numbers numerically instead of resolving them to hostnames and service names.
Block ping (inbound ICMP Echo Request):
- iptables -A INPUT -p icmp --icmp-type 8 -j DROP
- Here, -A INPUT appends a rule to the end of the INPUT chain (which handles packets destined for this machine). -p icmp specifies the ICMP protocol, --icmp-type 8 is ICMP type 8 (Echo Request), and -j DROP drops matching packets.
Save iptables rules (if you want them to persist):
- How to save iptables rules differs between Linux distributions.
- On RHEL-based systems like CentOS, use the service iptables save command. It writes the current rules to /etc/sysconfig/iptables so they survive a reboot.
- On Debian-based systems like Ubuntu, install the iptables-persistent package. After installation, use iptables-save > /etc/iptables/rules.v4 (for IPv4 rules) to save the rules so they’re restored after a reboot.
Restore ping

To restore ping, delete the rule you just added with the iptables -D INPUT -p icmp --icmp-type 8 -j DROP command. -D INPUT removes a rule from the INPUT chain; the other parameters are the same as when adding the rule.

Block Ping with UFW#

NOTE
To err on the side of caution, the rules related to destination-unreachable, time-exceeded, and parameter-problem have been removed.

UFW has no direct command for blocking the icmp protocol. UFW defines allow rules for ping in /etc/ufw/before.rules; change ACCEPT to DROP there to block ping:

1
-A ufw-before-input -p icmp --icmp-type echo-request -j DROP
2
-A ufw-before-forward -p icmp --icmp-type echo-request -j DROP

Heads-up: this only blocks IPv4 — the box is still pingable over IPv6.

To block ping over IPv6 as well, edit /etc/ufw/before6.rules:

1
-A ufw6-before-output -p icmpv6 --icmpv6-type echo-request -j DROP
2
-A ufw6-before-output -p icmpv6 --icmpv6-type echo-reply -j DROP

[!INFO] Leave the remaining icmpv6 rules unchanged.

Restrict SSH Access by IP#

My office network has a static IP, so I open port 22 in the VPS provider’s firewall and restrict it to that IP. Whenever I need access from another IP, I just add it.

If you have a dynamic public IP and your provider offers an API for modifying firewall rules, you can use that directly.

If you don’t have a static IP at home, you can use a web terminal like Tencent Cloud’s orcaterm — the basic features are free forever. Here are orcaterm’s IP ranges; load them into ipset and allow only those IPs to reach the SSH port. Test that the connection actually works before applying the rule. If you’re a Tencent Cloud user, you can even drop the SSH port entirely in the security group — when I used Tencent Cloud, I only opened ports 80 and 443.

You can also do this with UFW.

Or use the firewall provided by your VPS provider (if supported) — it’s easier to adjust when you run into connectivity problems.

Steps for adding servers from other providers to orcaterm:

Keep Software Updated#

Update Regularly#

My advice is to log into your VPS periodically and run sudo apt update && sudo apt upgrade to keep every package up to date.

That said, Ubuntu installs security updates automatically every day by default, so you don’t need to do this too often.

Enable Ubuntu Pro#

The same great OS, with more security updates Reducing the average CVE (Common Vulnerabilities and Exposures) exposure time from 98 days to 1 day Extended CVE patching, ten years of security maintenance, optional support, and maintenance for your entire open-source application stack

That’s Ubuntu’s official marketing copy. I can’t tell you exactly how useful it is, but a fix is better than no fix — and it’s free for up to 5 machines for personal use, so enabling it costs you nothing.

First, create an Ubuntu One account: https://ubuntu.com/login

After registering, go to https://ubuntu.com/pro/dashboard to find your token.

Once you have your token, head to the VPS and run sudo pro attach [YOUR_TOKEN]. After a short wait, Ubuntu Pro is enabled. I’d recommend running sudo apt update && sudo apt upgrade once more afterwards to make sure the latest security updates are installed.

Hide the Public IP#

Hiding the public IP isn’t a security requirement shared by every VPS user. One off-the-cuff idea for the future (once IPv6 is widely deployed) is to expose only the origin’s IPv6 address to the CDN — mass scanners like Censys would take a very long time to sweep it. You’d still want an allowlist, though.

Stop SSL Certificates from Leaking Your IP#

This section, “Stop SSL Certificates from Leaking Your IP,” is quoted from “How to Prevent Certificates from Leaking Your Origin IP” by Qiu Weimeng, published under the CC BY-SA 4.0 license. The rest of this article is published under the CC BY-NC-SA 4.0 license, but this section uses the CC BY-SA 4.0 license.

Apply for and Download a Certificate#

Sign up and log in to ZeroSSL
Go to the Dashboard, find “Create SSL Certificate,” and click the blue “New Certificate” button
Enter your origin IP under “Enter Domains”
Expiry doesn’t really matter — the point is to keep Censys from scanning a real certificate for your domain. So choose a 90-day certificate
Leave CSR & Contact unchanged
Choose “HTTP File Upload” as the domain validation method
- If you use NGINX and haven’t touched /etc/nginx/sites-available/default, you’re fine. Of course, if you’ve changed it, just make sure a server {listen 80; root /var/www/html;} block remains
- Then download the Auth File and upload it to /var/www/html/.well-known/pki-validation. Create the folder if it doesn’t exist, and make sure NGINX can access these files, or validation will still fail
- As prompted, click the .txt file to check whether it’s accessible. If it loads, validation is done

Configure the Certificate and Block IP-Based HTTP Access on Ports 80/443#

Download the *.zip file and extract it. The extracted folder contains the certificate you applied for
Merge ca_bundle.crt and certificate.crt: open certificate.crt in a reliable editor such as notepad3 or VSCode, then paste the contents of ca_bundle.crt into it. The format is:

1
-----BEGIN CERTIFICATE-----
2
contents of certificate.crt
3
-----END CERTIFICATE-----
4
-----BEGIN CERTIFICATE-----
5
contents of ca_bundle.crt
6
-----END CERTIFICATE-----

Upload the merged file and private.key
Upload the certificate to a folder NGINX can access — we put it in /etc/nginx/ip-certificate/
Configure /etc/nginx/sites-available/default. Reference config:

1
server {
2
#HTTP Server Default Set
3
    listen 80;
4
    listen 443 ssl http2 default_server;
5
    server_name ip;
6

7
    #HTTP_TO_HTTPS_END
8
    ssl_certificate /etc/nginx/ip-certificate/certificate.crt;
9
    ssl_certificate_key /etc/nginx/ip-certificate/private.key;
10
    ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
11
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
12
    ssl_prefer_server_ciphers on;
13
    ssl_session_cache shared:SSL:10m;
14
    ssl_session_timeout 10m;
15

16
    #Server ROOT
17
    index index.html;
18
    root /var/www/html/;
19
    index index.html;
20

21
    return 444; #NGINX HTPP Code 444
22
}

Run nginx -t on the server to check for errors; if it’s valid, restart with sudo systemctl restart nginx.

Type your IP directly into a browser and check whether the connection immediately turns into a blank page.

The Newer Method (Nginx 1.19.4+)#

If your Nginx version is 1.19.4 or newer, you can use the ssl_reject_handshake directive to reject every TLS handshake that doesn’t match a configured domain:

1
server {
2
    listen 443 ssl default_server;
3
    server_name _;
4
    ssl_reject_handshake on;
5
}

If you get an error like nginx: [emerg] unknown directive "ssl_reject_handshake", first check whether your Nginx version is at least 1.19.4 — ssl_reject_handshake on; was added in Nginx 1.19.4 (mainline).

If you compiled Nginx from source, make sure the ngx_http_ssl_module module is enabled. It isn’t built by default; you need the --with-http_ssl_module configure option.

Are We Safe Now?#

In the previous sections, we only ensured that an attacker can’t fetch a default certificate by visiting your IP directly and infer your domain from it. Nothing says an attacker has to learn IP-to-domain mappings that way, though. As you can see, the rules above all rely on server_name matching. An attacker could easily walk through every non-CDN IP range carrying the correct server_name, recording which targets respond correctly. Here’s a simple implementation of that check (without the scanning part):

1
# https://gist.github.com/Raven95676/39ffdba22144e39d7155ad9dc1bcca55
2
import ssl
3
import socket
4
def check (ip, domain):
5
    try:
6
        context = ssl.create_default_context ()
7
         with socket.create_connection ((ip, 443)) as sock:
8
            with context.wrap_socket (sock, server_hostname=domain) as ssl_sock:
9
                request = f"GET / HTTP/1.1\nHost: {domain}\nConnection: close\n\n"
10
                ssl_sock.sendall (request.encode ())
11
                ssl_sock.recv (4096)
12
        return True
13
    except Exception:
14
        return False
15

16
print (check ("192.168.0.256", "example.com"))

Patching It Up#

TIP
If your VPS provider offers a firewall, you can just use the provider’s firewall.

You can just use this method

We can allow only the CDN’s CIDR ranges to reach ports 80/443 on the server. First, add the allow rules:

1
sudo ufw allow from "CIDR range" to any proto tcp port 80,443 comment "CDN provider"

The comment isn’t required — it’s just so you can recognize what the rule is for later.

Ask your CDN provider for its CIDR ranges — some publish them on their website. For example, Cloudflare’s CIDR ranges are available at: https://www.cloudflare.com/ips-v4

That’s the IPv4 list; the IPv6 one is at: https://www.cloudflare.com/ips-v6

For Cloudflare, a community member also wrote a script that syncs Cloudflare’s IPs into UFW for port 443 when run:

1
RULES=$(sudo ufw status numbered | grep 'Cloudflare IP' | awk -F"[][]" '{print $2}' | sort -nr)
2
for RULE in $RULES; do
3
    echo "Deleting rule $RULE"
4
    echo "y" | sudo ufw delete $RULE
5
done
6

7
for cfip in `curl -sw '\n' https://www.cloudflare.com/ips-v {4,6}`; do ufw allow proto tcp from $cfip to any port 443 comment 'Cloudflare IP'; done
8

9
ufw reload > /dev/null

Because of the special circumstances involved, I’d suggest running the following command first to check that the output is correct before deploying the script:

1
for cfip in `curl -sw '\n' https://www.cloudflare.com/ips-v {4,6}`; do echo $cfip; done

If the output looks right (like below), you can deploy it:

1
173.245.48.0/20
2
103.21.244.0/22
3
… (omitted)
4
2400:cb00::/32
5
2606:4700::/32
6
… (omitted)

After adding the rules, list the current firewall rules:

1
sudo ufw status numbered

Delete any pre-existing rules that allow all traffic on ports 80 and 443:

1
sudo ufw delete <number>

Why not just use Nginx’s deny directive? Because deny returns a 403 Forbidden status code — and before it can, the TLS handshake must complete. Only after a successful TLS handshake can the client send an HTTP request and receive a response. So using deny directly here would be doing pointless work.

Are We Really Safe…?#

This is really a Cloudflare special, but the ideas apply to other CDN providers too if you want extra security. Cloudflare offers more than CDN services — there’s a whole lineup of products like Workers and WARP. Some of these services have characteristics worth noting:

They can make outbound requests
They use Cloudflare’s IP ranges

Cloudflare certainly restricts abuse, but just in case, we can take one more security measure — Authenticated Origin Pulls.

[!INFO] Make sure the SSL/TLS encryption mode is set to Full or Full (strict).

Download the Cloudflare certificate and configure it:

1
ssl_client_certificate /path/to/certificate.pem;
2
ssl_verify_client on;

Then enable Authenticated Origin Pulls under SSL/TLS → Origin Server.

The SafeLine WAF community edition can’t stably use this method yet, unless you’re willing to put another reverse proxy in front of it.

Moving Toward Full Containerization#

My advice: make good use of containerization for isolation.

This section doesn’t cover server panels like 1Panel or Docker panels like Portainer.

Docker Basics#

NOTE
There’s mature documentation available, so I won’t reinvent the wheel.
This article is unrelated to the https://github.com/yeasy/docker_practice project.
Just being a little lazy

Side note: if docker-compose says the command isn’t found, try docker compose. The version field is deprecated in docker compose — it will be ignored with a warning in existing compose files.

https://yeasy.gitbook.io/docker_practice

Update: added the official Chinese Docker documentation.

https://docs.docker.net.cn/manuals/

When UFW Can’t Control Docker (Recommended)#

One more thing about Docker: you don’t really need UFW to manage which ports Docker exposes — Docker writes its own iptables rules to control them. Using docker compose as an example:

1. For Services Used Only Within the App (Databases, Redis, etc.)#

Only expose them on the internal container network. When compose brings up the stack, it creates a bridge named <first-service-name>_default, which you can see with docker network ls.
Containers expose all their ports on this internal network by default.
Other containers on the same network can reach the service by container name and port.
Containers not attached to that network can’t access any of its ports.
For services only exposed on the internal network, no port mapping is needed under ports.

2. For Services Behind a Reverse Proxy#

Binding to 127.0.0.1 only keeps the port from being reachable from outside; use this on the container in question:

1
ports:
2
    - 127.0.0.1:<port>:<port>

3. For Services That Need Direct External Port Access#

Map the port directly:

1
ports:
2
      - <port>:<port>

And here’s an example compose.yml:

1
services:
2
    rsshub:
3
        image: diygod/rsshub:latest
4
        restart: always
5
        ports:
6
            - 127.0.0.1:1200:1200
7
        environment:
8
            NODE_ENV: production
9
            CACHE_TYPE: redis
10
            REDIS_URL: "redis://redis:6379/"
11
            PUPPETEER_WS_ENDPOINT: "ws://browserless:3000"
12
        env_file:
13
            - .env
14
        healthcheck:
15
            test: ["CMD", "curl", "-f", "http://localhost:1200/healthz"]
16
            interval: 30s
17
            timeout: 10s
18
            retries: 3
19
        depends_on:
20
            - redis
21
            - browserless
22

23
    browserless:
24
        image: browserless/chrome
25
        restart: always
26
        ulimits:
27
            core:
28
                hard: 0
29
                soft: 0
30
        healthcheck:
31
            test: ["CMD", "curl", "-f", "http://localhost:3000/pressure"]
32
            interval: 30s
33
            timeout: 10s
34
            retries: 3
35

36
    redis:
37
        image: redis:alpine
38
        restart: always
39
        volumes:
40
            - ./data:/data
41
        healthcheck:
42
            test: ["CMD", "redis-cli", "ping"]
43
            interval: 30s
44
            timeout: 10s
45
            retries: 5
46
            start_period: 5s
47

48
    sb:
49
        image: ghcr.io/sagernet/sing-box
50
        container_name: sb
51
        restart: always
52
        volumes:
53
            - ./sing-box:/etc/sing-box/
54
        command: -D /var/lib/sing-box -C /etc/sing-box/run

Additionally, if you need to keep external ports blocked, besides binding to 127.0.0.1 there’s another option:

For services that need a reverse proxy (Nginx/Caddy, etc.), configure a Docker-private IP starting with 172 in the Compose file and don’t configure a port. This way Docker won’t expose any public port, and the reverse proxy can just point at the private IP and its default port.

For example, this MySQL Compose file assigns the private IP 172.20.0.15 and comments out the port mapping; when configuring the reverse proxy, simply proxy to 172.20.0.15:3306:

1
networks:
2
  default:
3
    external: true
4
    name: ${DOCKER_MY_NETWORK}
5
services:
6
  mysql:
7
    container_name: mysql8
8
    image: mysql:8
9
    # ports:
10
    #   - "3306:3306"
11
    environment:
12
      TZ: Asia/Shanghai
13
    networks:
14
      default:
15
        ipv4_address: 172.20.0.15
16
    restart: unless-stopped

When UFW Can’t Control Docker (Not Recommended)#

This section, “When UFW Can’t Control Docker (Not Recommended),” is quoted from the README of the project ufw-docker by chaifeng, released under the GPL-3.0 license. The rest of this article is published under the CC BY-NC-SA 4.0 license, but this section uses the GPL-3.0 license.

The current new solution only requires modifying one UFW config file; keep all of Docker’s own config and options at their defaults.

Edit UFW’s config file /etc/ufw/after.rules and append the following rules:

1
# BEGIN UFW AND DOCKER
2
*filter
3
:ufw-user-forward - [0:0]
4
:ufw-docker-logging-deny - [0:0]
5
:DOCKER-USER - [0:0]
6
-A DOCKER-USER -j ufw-user-forward
7

8
-A DOCKER-USER -j RETURN -s 10.0.0.0/8
9
-A DOCKER-USER -j RETURN -s 172.16.0.0/12
10
-A DOCKER-USER -j RETURN -s 192.168.0.0/16
11

12
-A DOCKER-USER -p udp -m udp --sport 53 --dport 1024:65535 -j RETURN
13

14
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 192.168.0.0/16
15
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 10.0.0.0/8
16
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 172.16.0.0/12
17
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 192.168.0.0/16
18
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 10.0.0.0/8
19
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 172.16.0.0/12
20

21
-A DOCKER-USER -j RETURN
22

23
-A ufw-docker-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW DOCKER BLOCK]"
24
-A ufw-docker-logging-deny -j DROP
25

26
COMMIT
27
# END UFW AND DOCKER

Then restart UFW with sudo systemctl restart ufw. From now on, the outside can no longer access any port published by Docker, while containers can still communicate with each other on private network addresses, and containers can still access the external network normally. The rules may fail to take effect after restarting UFW due to some unknown reason — if so, reboot the server.

To allow external access to a service provided by a Docker container — say a container’s service port is 80 — use the following command:

1
ufw route allow proto tcp from any to any port 80

This allows external access to every service published by Docker whose internal service port is 80.

Note that this 80 is the container’s port, not the 8080 port published on the server with something like -p 0.0.0.0:8080:80.

If multiple containers have service port 80 but you only want a specific one reachable from outside — say its private address is 172.17.0.2 — use a command like:

1
ufw route allow proto tcp from any to 172.17.0.2 port 80

If a container’s service uses UDP, say a DNS service, use the following command to allow external access to all published DNS services:

1
ufw route allow proto udp from any to any port 53

Similarly, to target just one specific container, e.g. the IP 172.17.0.2:

1
ufw route allow proto udp from any to 172.17.0.2 port 53

Chaitin SafeLine WAF + Cloudflare Single-Node Deployment#

After racking my brain, there’s not much to write

If you need stronger security, stack a WAF and a honeypot in front like I did.

Deployment#

There’s a one-click deployment command:

1
bash -c "$(curl -fsSLk https://waf-ce.chaitin.cn/release/latest/setup.sh)"

Or deploy the LTS version:

1
RELEASE=lts bash -c "$(curl -fsSLk https://waf-ce.chaitin.cn/release/latest/setup.sh)"

Since this setup puts Cloudflare in front of the SafeLine WAF, you need to set Protected Sites → Global Config → Origin IP retrieval to “Take the address of the previous proxy from X-Forwarded-For”.

If other issues come up, check the troubleshooting section of the official documentation.

Accessing the SafeLine Console via a Domain#

Best solution: don’t use a domain

If you want to access the SafeLine console via a domain, you’ll find it doesn’t work once Cloudflare’s orange cloud is enabled. That’s because Cloudflare’s port forwarding only supports a fixed set of ports:

HTTP ports supported by Cloudflare: 80, 8080, 8880, 2052, 2082, 2086, 2095

HTTPS ports supported by Cloudflare: 443, 2053, 2083, 2087, 2096, 8443

The solution is simple: go to Rules → Origin Rules and create a rule. The incoming request match expression looks roughly like:

1
(starts_with (http.request.full_uri, "https://<your-domain>"))

Set the destination port rewrite to 9443.

To hide your public IP, add a firewall rule that only lets Cloudflare’s CDN CIDR ranges reach port 9443.

Redirect Issues#

If a port number unexpectedly appears in the URL after a redirect, add the following under Protected Sites → Site Details → Custom NGINX Config:

1
proxy_redirect https://$host:[port] https://$host;

Hide the Admin Panel with WireGuard#

Don’t expose your VPS admin panels (such as the SafeLine WAF or BT Panel) to the public internet. Accessing them over a VPN like WireGuard is a more secure choice. Detailed WireGuard installation instructions are available at Installation - WireGuard.

Raven Note

Setting up Tailscale and configuring HTTPS on Synology NAS: A practical guide

Setting up Tailscale#

Setting up HTTPS and automating it#

References#

Python from Beginner to Advanced: Classes, Objects, and Object-Oriented Programming

OOP: A Prologue#

Conventions#

Basic Concepts#

Magic Methods#

Decorators#

Namespaces#

Scope and Closures#

A First Look at Classes and Objects#

Reuse Through Inheritance#

Do We Really Need Inheritance?#

Customizing Class Behavior#

With Magic Methods#

Customizing Instantiation with __new__#

Custom String Representations#

Comparisons with Magic Methods#

Context Managers#

Optimizing Attribute Access with slots#

Iterators and Generators#

With Metaclasses#

Introduction#

The Python Object System — From the Ground Up#

Using Metaclasses#

Practical OOP Tips in Python#

”Private” Attributes#

Defining Interfaces with Abstract Base Classes#

Using Enums#

Eliminating Duplicate Code with Decorators#

Writing Your Own Decorators#

Using the lru_cache Decorator#

Ensuring Unique Enum Values with @unique#

Using the property Decorator#

Using the dataclass Decorator#

Metaclass Alternatives#

Class Decorators Instead of Metaclasses#

Magic Methods Instead of Metaclasses#

When Only a Metaclass Will Do#

OOP Best Practices#

KISS#

YAGNI#

SOLID#

S - Single Responsibility Principle#

O - Open/Closed Principle#

L - Liskov Substitution Principle#

I - Interface Segregation Principle#

D - Dependency Inversion Principle#

DRY#

Summary#

References#

Useful Registry Keys (Continuously Updated)

Auto-start#

User-level#

System-level#

Image File Execution Options#

Image hijacking#

SilentProcessExit#

MinimumStackCommitInBytes#

Scheduled Tasks#

Hierarchy information#

Task details#

Deleting Files After Reboot#

Unlocking BitLocker from the Command Line

Basic Security Measures for a VPS

Preface#

Account Security#

Create a Non-root Account#

Disable Root Password Login over SSH#

Change the SSH Port#

Protect SSH from Brute Force with Fail2ban#

SSH Login Notifications#

[Strict] Run Commands with Absolute Paths#

Log In with SSH Keys#

Enable the UFW Firewall#

Block Ping#

Block Ping with iptables#

Customizing Instantiation with `new`#