Logging with python
0. Intro
Disclaimer: A lot has changed since I wrote this. Right now I wouldn’t recommed you to follow my steps.
Instead I’d suggest you use something like Loguru. I’ll still keep this here for the record.
In almost every program that we write we will need to show some output. The first thing that developers use is the print function but this lacks some really important features such as providing the time when the message was produced and the ability to save the output to a file. This is way is better to use a logging.
1. pygogo
As always the first thing you should do when you need something is to look at the existing packages to avoid reinventing the wheel. Pygogo is a nice python package that is easy to use.
The key features I think it has are:
- You can write regular message (low) in one file and error-like messages (high) in another one
- It can have different formats for
lowandhigh - It can be called from different files and handles well the different paths
What I belive it lacks:
- Ability to show full information of errors
- Easy way to have different formats for console handlers and file handlers.
So I decided to create my own logging library
2. v-log
What I wanted was a log that would output good looking messages to the console while writting a properly formatted csv for further analysis. It is difficult to achive that with the regular logging library so I created v-log.
2.1. Using v-log
To install it run:
pip install v-log
It works similar to the default log:
from v_log import VLogger
log = VLogger(__name__)
log.critical("critical")
log.error("error")
log.warning("warning")
log.info("info")
log.debug("debug")
It can also show execution times and errors:
# Show time
log.info("Test time", time=10)
# Show error. You need to pass the exception with the 'error' param
try:
1 / 0
except Exception as e:
log.error("Try errors", error=e)
log.error("Try errors %s", "full", time=10, error=e)
This is what you will see with the above example:
Output in the console will have colors. debug is green, info the default color, warning is orange and both error and critical are red.
2.2. How it works
v-log creates a VLogger class that is a wrapper that has 2 logs inside, one for the file output and another one for the terminal. This way if you call log.info it will call both log_terminal.info and log_console.info functions.
The second intersting part is that you can pass time as and argument and it will process it. You can also pass an exception and it will show the line where the error happened, the type of error and the details.
2.3. Customizating v-log
When creating the VLogger instance you can define:
| param | description | default |
|---|---|---|
| module_name | name of the module | "base" |
| uri_log | uri of the file where log will be stored | "log.csv" |
| file_log_level | minimum level of log events in order to be writed | logging.INFO |
| console_log_level | minimum level of log events in order to be printed | logging.INFO |
| csv_separator | csv separator | ";" |
| base_path | name of the root folder of the execution | "src" |
for example you could do:
import logging
from v_log import VLogger
log = VLogger(__name__, uri_log="data/log.log", file_log_level=logging.WARNING)
