When a program runs on your computer you can watch it, stop it and inspect it with the debugger. When it runs on a server, at three in the morning, serving hundreds of users, the only window onto what is happening is the logs. A good logging system turns an hours-long investigation into a search of minutes; a bad one produces gigabytes of text in which the useful information cannot be found.
Logs and prints are not the same thing
Many people start with print() calls scattered through the code. It works for an experiment, but in a real application its limits show immediately: you cannot filter by importance, it does not say when or where the message was written, and it cannot be routed to a file or a central service without changing the code. Logging libraries, such as the logging module in Python's standard library, solve these problems by separating what is recorded from where and how it is recorded.
import logging
logger = logging.getLogger(__name__)
def import_customers(path):
logger.info("Starting import from %s", path)
...
Using getLogger(__name__) gives each module a logger with its own name, so the logs immediately show which part of the code a message comes from.
Severity levels
Every message has a level. Using them consistently is what makes logs filterable:
| Level | When to use it | Example |
|---|---|---|
| DEBUG | Details useful only during development or an investigation | Intermediate values of a calculation |
| INFO | Normal, meaningful events | Service started, import completed |
| WARNING | Something unexpected, but the program continues | Retry after a timeout |
| ERROR | An operation failed | Unable to save an order |
| CRITICAL | The system cannot keep running | Database unreachable at startup |
In production you usually record from INFO upwards; DEBUG is switched on temporarily when needed. A frequent mistake is logging everything as ERROR: after a few days nobody looks at errors any more, because they are noise.
Messages with context
A message like "Error while saving" is almost useless. Saving what? For whom? A good log message answers the questions you will ask yourself when you read it:
logger.error(
"Order save failed: order=%s customer=%s amount=%.2f",
order.id, order.customer_id, order.total,
)
Include the identifiers that let you link the message to the data: order ID, user ID, request ID. In a web application it is very useful to assign each request a unique identifier and include it in every message about it: that way you can reconstruct the entire path of a single request among tens of thousands of lines.
Note the form too: with the logging module it is preferable to pass parameters separately instead of building the string in advance. Formatting only happens if the message is actually recorded.
Logging exceptions
Inside an except block, logger.exception() records the message at ERROR level and automatically adds the full traceback:
try:
send_invoice(order)
except SendError:
logger.exception("Invoice sending failed for order %s", order.id)
raise
Beware of a harmful habit: logging the same exception at every level as it propagates. The result is the same error repeated five times, which confuses the reader. Log the error once, at the point where it is handled.
Structured logs
Free-text logs are convenient to read by eye but hard to analyse with automated tools. Structured logs record each event as a set of fields, usually in JSON format:
{"ts": "...", "level": "ERROR", "event": "order_save_failed",
"order": 4821, "customer": 77, "duration_ms": 1530}
With a format like this, a log collection system can filter all events for one customer, count errors by type or compute average times. There are dedicated libraries, such as structlog for Python, but the standard module can also produce JSON with a custom formatter.
What never to record
Logs get copied, retained, sent to external services and read by many people. That is why some information must never end up in them:
- passwords, access tokens, API keys;
- full payment card numbers and security codes;
- the full content of private documents or messages;
- personal data not needed for the investigation, also to respect the data minimisation principle set out in the GDPR.
A typical case is logging the entire body of an HTTP request for convenience: along with the useful data, credentials end up in the logs too. It is better to record only the fields you need, or mask the sensitive ones.
Configuration separate from code
The code should only emit messages; where they go, in what format and from what level is a configuration choice. In development you may want coloured terminal output at DEBUG level, in production JSON at INFO level sent to a central system. Keeping this configuration in a single place, at application startup, avoids having to edit dozens of files.
In short
- Use a logging library instead of
print, with one logger per module. - Apply levels consistently: not everything is an error.
- Write messages with context and identifiers, such as the request or order ID.
- Log exceptions only once, with the traceback, and consider structured JSON logs.
- Never record passwords, tokens, payment data or unnecessary personal data.
