Skip to content

Sqlparser Binary Size #2514

Description

@liningpan

It appears that sqlparser is contributing significantly to DataFusion and downstream projects that uses DataFusion as a library. apache/datafusion#24727 (comment)

cargo bloat suggests that 6.4MB/100MB of the datafusion-cli binary (release build) is attributed to sqlparser (for my own project, it bloats up to 20 MB/150MB).

> cargo bloat --bin datafusion-cli --release --filter sqlparser -n 100 --full-fn

File .text    Size     Crate Name
0.0%  0.1% 33.8KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_alter
0.0%  0.1% 32.8KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_create
0.0%  0.0% 26.7KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<hashbrown[a17ea828e164abc8]::hasher::DefaultHasher>
0.0%  0.0% 26.7KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<hashbrown[a17ea828e164abc8]::hasher::DefaultHasher>
0.0%  0.0% 26.7KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<hashbrown[a17ea828e164abc8]::hasher::DefaultHasher>
0.0%  0.0% 26.6KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_alter_table_operation
0.0%  0.0% 19.8KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::ddl::CreateTable as core[663bb9e1522ba5ef]::hash::Hash>::hash::<hashbrown[a17ea828e164abc8]::hasher::DefaultHasher>
0.0%  0.0% 19.8KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::ddl::CreateTable as core[663bb9e1522ba5ef]::hash::Hash>::hash::<hashbrown[a17ea828e164abc8]::hasher::DefaultHasher>
0.0%  0.0% 19.8KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::ddl::CreateTable as core[663bb9e1522ba5ef]::hash::Hash>::hash::<hashbrown[a17ea828e164abc8]::hasher::DefaultHasher>
0.0%  0.0% 19.4KiB sqlparser <sqlparser[f91aac8db406c26f]::dialect::snowflake::SnowflakeDialect as sqlparser[f91aac8db406c26f]::dialect::Dialect>::parse_statement
0.0%  0.0% 18.4KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_table_factor::{closure#0}
0.0%  0.0% 17.5KiB sqlparser sqlparser[f91aac8db406c26f]::dialect::snowflake::parse_create_table
0.0%  0.0% 17.4KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::fmt::Display>::fmt
0.0%  0.0% 17.4KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_create_table
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 17.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::clone::Clone>::clone
0.0%  0.0% 15.3KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_optional_create_table_config
0.0%  0.0% 15.0KiB sqlparser <sqlparser[f91aac8db406c26f]::tokenizer::Tokenizer>::next_token
0.0%  0.0% 14.7KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_prefix::{closure#1}
0.0%  0.0% 13.9KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.9KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.9KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.9KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::Statement as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.6KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_grant_deny_revoke_privileges_objects
0.0%  0.0% 13.2KiB sqlparser <sqlparser[f91aac8db406c26f]::parser::Parser>::parse_select
0.0%  0.0% 13.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::query::TableFactor as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::query::TableFactor as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::query::TableFactor as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>
0.0%  0.0% 13.1KiB sqlparser <sqlparser[f91aac8db406c26f]::ast::query::TableFactor as core[663bb9e1522ba5ef]::hash::Hash>::hash::<std[5d97c59e5e5fafcc]::hash::random::DefaultHasher>

It appears that there are a substantial number of hash and clone implementations that got inlined for Statement and CreateTable types. In my case, I got like 20 copies for Statement with hashbrown hasher alone.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions