Cannot init the db for Airflow

Question

I have some experience starting starting up Apache Airflow but I have now an error when I try to airflow db init command. The error is as below. I am running Airflow on virtual env with Python 3.8. Any help would appreciated. I am not sure to understand this error as I managed to init the db without importing any _cffi_backend module in the past.

The error:

airflow) airflow@26c8ed88c008:~/airflow$ airflow db init
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 47, in command
    func = import_string(import_path)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/module_loading.py", line 32, in import_string
    module = import_module(module_path)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py", line 24, in <module>
    from airflow.utils import cli as cli_utils, db
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/db.py", line 26, in <module>
    from airflow.jobs.base_job import BaseJob  # noqa: F401
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/__init__.py", line 19, in <module>
    import airflow.jobs.backfill_job
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/backfill_job.py", line 29, in <module>
    from airflow import models
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/__init__.py", line 20, in <module>
    from airflow.models.baseoperator import BaseOperator, BaseOperatorLink
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 59, in <module>
    from airflow.models.taskinstance import Context, TaskInstance, clear_task_instances
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 57, in <module>
    from airflow.models.variable import Variable
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/variable.py", line 24, in <module>
    from cryptography.fernet import InvalidToken as InvalidFernetToken
  File "/usr/lib/python3/dist-packages/cryptography/fernet.py", line 17, in <module>
    from cryptography.hazmat.primitives import hashes, padding
  File "/usr/lib/python3/dist-packages/cryptography/hazmat/primitives/padding.py", line 13, in <module>
    from cryptography.hazmat.bindings._padding import lib
ModuleNotFoundError: No module named '_cffi_backend'

score 0 · Answer 1 · answered Aug 04 '21 at 19:04

Airflow 2 has hard requirement on cryptography module being installed and operating. So you have to install it properly. There are some distributions of Linux that you need to add some additional packages to make it works.

You can take a look at this query and similar problems and possibly you will find out how cryptography can be installed or reinstalled cleanly.

No module named _cffi_backend

score 0 · Answer 2 · answered Aug 05 '21 at 16:19

Check the installation docs for cryptography: https://cryptography.io/en/latest/installation/

If you are using Alpine, make sure you install the following:

$ sudo apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo

Similarly for Debian/buster:

$ sudo apt-get install build-essential libssl-dev libffi-dev \
    python3-dev cargo

Check their installation guide for other distros and run airflow db init again

Cannot init the db for Airflow

2 Answers2