Set the value of column when upgrading table with alembic

Question

I am using PostgreSQL and Alembic for migration. When I added new column to my User table Alembic generated migration with the following script:

revision = '4824acf75bf3'
down_revision = '2f0fbdd56de1'

from alembic import op
import sqlalchemy as sa

def upgrade():
    op.add_column(
        'user', 
        sa.Column(
            'username', 
            sa.Unicode(length=255), 
            nullable=False
        )
    )

def downgrade():
    op.drop_column('user', 'username')

What I actually want to do is autogenerating the value of username when I upgrade the production version. By other words, my production version has lots of users in it and if I run the above upgrade on it, there will be an error stating that the username cannot be NULL, so I have to delete all the users, upgrade the User table and after add the users again, which is painful. Thus, I want to change the above script with:

revision = '4824acf75bf3'
down_revision = '2f0fbdd56de1'

from alembic import op
import sqlalchemy as sa

def upgrade():
    op.add_column(
        'user', 
        sa.Column(
            'username', 
            sa.Unicode(length=255)
        )
    )
    op.execute(
        'UPDATE "user" set username = <email address with no '@' 
         and everything comes after '@' sign should be removed> 
         WHERE email is not null'
    )
    <only after the above code is executed 'nullable=False' must be set up>

def downgrade():
    op.drop_column('user', 'username')

As it is stated above in the code I would like to execute a SQL Code that checks the email address like test@example.com and throws everything after '@'sign (in this case '@example.com') and sets the value of username (in this case 'test') after that makes nullable=false.

How can I do that? What must be the script instead of username = <email address with no '@' and everything comes after '@' sign should be removed> and setting nullable=false

Or if there is any other way of setting username default value to be email address without @ sing and everything after it?

Thanks for asking this; wish the alembic instructions were more clear that op.execute is the way to go about this. — Job Evers, Dec 24 '14 at 08:16
Maybe it can help: https://medium.com/the-andela-way/alembic-how-to-add-a-non-nullable-field-to-a-populated-table-998554003134 — Evgenii, Dec 21 '21 at 20:41

score 7 · Accepted Answer · answered Aug 26 '14 at 12:33

here is how the issue was solved.

def upgrade():
    op.add_column(
        'user',
        sa.Column(
            'username',
            sa.Unicode(length=255)
        )
    )
    op.create_index('ix_user_username', 'user', ['username'], unique=True)
    op.execute(
        '''
        DO
        $do$
        DECLARE uid INTEGER;
        DECLARE username_candidate TEXT;
        BEGIN
        FOR uid, username_candidate IN (
            SELECT
                id,
                lower(
                    substring(email for position('@' in email) - 1)
                )
            FROM "user" WHERE username is null
        ) LOOP
            UPDATE "user"
            SET username = username_candidate
            WHERE
                id = uid AND
                NOT EXISTS (
                SELECT id FROM "user" WHERE username = username_candidate
            );
        END LOOP;
        END
        $do$
        '''
    )
    # Fix name colissions
    op.execute(
        '''
        DO
        $do$
        DECLARE uniqufier INTEGER := 0;
        DECLARE uid INTEGER;
        DECLARE username_candidate TEXT;
        BEGIN
        WHILE EXISTS (SELECT id FROM "user" WHERE username is null) LOOP
            uniqufier := uniqufier + 1;
            FOR uid, username_candidate IN (
                SELECT
                    id,
                    lower(
                        substring(email for position('@' in email) - 1)
                        || uniqufier
                    )
                FROM "user" WHERE username is null
            ) LOOP
                UPDATE "user"
                SET username = username_candidate
                WHERE
                    id = uid AND
                    NOT EXISTS (
                        SELECT id FROM "user" WHERE username = username_candidate
                    );
            END LOOP;
        END LOOP;
        END;
        $do$
        '''
    )
    op.alter_column(
        'user',
        'username',
        nullable=False,
    )


def downgrade():
    op.drop_index('ix_user_username', table_name='user')
    op.drop_column('user', 'username')

Tim · Answer 2 · 2018-05-14T10:33:41.530

0

The script to update a new column based on values from another column of the same table can be written with a subquery. The only trick is, because you query from same table, you will need to give aliases to table name to ensure you select from corresponding rows:

update 'user' as target set username = (
  select substring(email from '.+?(?=@)')
  from 'user' as source where source.id = target.id
);

edited May 14 '18 at 10:33

answered May 14 '18 at 10:19

Tim

12,318
7
50
72

Set the value of column when upgrading table with alembic

2 Answers2