db.model schema - string column without size to default to length 255

[ntai-arxiv]

(1) db.models - When a column type is Python "str" or String type without max length, default to 255. Otherwise, DB other than sqlite3 cannot have VARCHAR without size.

(2) Add the schema deployment to MySQL in github action tests so we'd catch the db.model change that doesn't work.

[bdc34]

Please do not set the string length to 255 by default. This will be incorrect.

[bdc34]

See this file for the lengths of strings to use for in models: https://github.com/arXiv/modapi/blob/develop/modapi/tables/arxiv_models.py

[ntai-arxiv]

Please do not set the string length to 255 by default. This will be incorrect.

@bdc34 arxiv_model.py uses Column(Text) which has no size which also fails. So, if not 255, what does it use for default? I guess "str" has no default length.

Following ones

Text: table arXiv_bib_feeds column prune_ids
Text: table arXiv_bib_feeds column prune_regex

for example has no size. I think it is defaulting to 255 from the DDL.
I think Text default size is VARCHAR 255, looking at DDL.

Ones failing is "str" and if I replace it with Text, it would be the same.

class BibFeeds(Base):
    __tablename__ = 'arXiv_bib_feeds'

    bib_id = Column(MEDIUMINT(8), primary_key=True)
    name = Column(String(64), nullable=False, server_default=text("''"))
    priority = Column(TINYINT(1), nullable=False, server_default=text("'0'"))
    uri = Column(String(255))
    identifier = Column(String(255))
    version = Column(String(255))
    strip_journal_ref = Column(TINYINT(1), nullable=False, server_default=text("'0'"))
    concatenate_dupes = Column(INTEGER(11))
    max_updates = Column(INTEGER(11))
    email_errors = Column(String(255))
    prune_ids = Column(Text)
    prune_regex = Column(Text)
    enabled = Column(TINYINT(1), server_default=text("'0'"))

-- auto-generated definition
create table arXiv_bib_feeds
(
    bib_id            mediumint(8) auto_increment
        primary key,
    name              varchar(64) default '' not null,
    priority          tinyint(1)  default 0  not null,
    uri               varchar(255)           null,
    identifier        varchar(255)           null,
    version           varchar(255)           null,
    strip_journal_ref tinyint(1)  default 0  not null,
    concatenate_dupes int                    null,
    max_updates       int                    null,
    email_errors      varchar(255)           null,
    prune_ids         text                   null,
    prune_regex       text                   null,
    enabled           tinyint(1)  default 0  null
);

it is probably more precise to auto-gen the models from DDL.

[norbusan]

@bdc34 arxiv_model.py uses Column(Text) which has no size which also fails. So, if not 255, what does it use for default? I guess "str" has no default length.

sqlalchemy's text translates to mysql's text which is a an arbitrarily sized character array, like a varchar without maximum size. So it might initially be initialized to 255 chars, but can grow.

for example has no size. I think it is defaulting to 255 from the DDL. I think Text default size is VARCHAR 255, looking at DDL.

I don't think so:

• Column(Text()) translates to a text entry in mysql
• mysql text: https://dev.mysql.com/doc/refman/8.4/en/blob.html