diff --git a/_data/documentation.yml b/_data/documentation.yml index 3f75bc4..08e8ef1 100644 --- a/_data/documentation.yml +++ b/_data/documentation.yml @@ -64,6 +64,10 @@ docs: url: "/tutorial/get-started-telemetry" description: "Learn on how to start capturing and publishing RepoDB operation telemetry using RepoDb.Telemetry.Default." + - title: "Vertica" + url: "/tutorial/get-started-vertica" + description: "Learn on how to work with Vertica databases using RepoDB library." + # - title: "Installation" # url: "/tutorial/installation" @@ -1008,6 +1012,12 @@ docs: - title: "Firebird (Bulk)" url: "/release/firebirdbulk" + - title: "Vertica" + url: "/release/vertica" + + - title: "Vertica (Bulk)" + url: "/release/verticabulk" + - title: "Telemetry (Core)" url: "/release/telemetry-core" diff --git a/pages/attributes/vertica/sourcecolumn.md b/pages/attributes/vertica/sourcecolumn.md new file mode 100644 index 0000000..729dc2f --- /dev/null +++ b/pages/attributes/vertica/sourcecolumn.md @@ -0,0 +1,51 @@ +--- +layout: default +title: SourceColumn +permalink: /attribute/vertica/sourcecolumn +tags: [repodb, attribute, sourcecolumn] +parent: "Vertica" +grand_parent: ATTRIBUTES +--- + +# SourceColumn + +--- + +This attribute sets the `VerticaParameter.SourceColumn` property value via a class property. + +### Attribute + +Example usage: + +```csharp +public class Person +{ + public int Id { get; set; } + + [SourceColumn("Name")] + public string Name { get; set; } +} +``` + +### Fluent Mapping + +To configure via [FluentMapper](/mapper/fluentmapper): + +```csharp +FluentMapper + .Entity() + .PropertyValueAttributes(e => e.Name, new SourceColumnAttribute("Name")); +``` + +### Retrieval + +Retrieve the attribute via [PropertyValueAttributeCache](/cacher/propertyvalueattributecache): + +```csharp +var attribute = PropertyValueAttributeCache + .Get(e => e.Name)? + .FirstOrDefault(e => e.GetType() == typeof(SourceColumnAttribute)); +``` + +{: .important } +> We strongly recommend using [PropertyValueAttributeCache](/cacher/propertyvalueattributecache) for maximum performance. diff --git a/pages/attributes/vertica/sourcecolumnnullmapping.md b/pages/attributes/vertica/sourcecolumnnullmapping.md new file mode 100644 index 0000000..c548cb8 --- /dev/null +++ b/pages/attributes/vertica/sourcecolumnnullmapping.md @@ -0,0 +1,51 @@ +--- +layout: default +title: SourceColumnNullMapping +permalink: /attribute/vertica/sourcecolumnnullmapping +tags: [repodb, attribute, sourcecolumnnullmapping] +parent: "Vertica" +grand_parent: ATTRIBUTES +--- + +# SourceColumnNullMapping + +--- + +This attribute sets the `VerticaParameter.SourceColumnNullMapping` property value via a class property. + +### Attribute + +Example usage: + +```csharp +public class Person +{ + public int Id { get; set; } + + [SourceColumnNullMapping(true)] + public string Name { get; set; } +} +``` + +### Fluent Mapping + +To configure via [FluentMapper](/mapper/fluentmapper): + +```csharp +FluentMapper + .Entity() + .PropertyValueAttributes(e => e.Name, new SourceColumnNullMappingAttribute(true)); +``` + +### Retrieval + +Retrieve the attribute via [PropertyValueAttributeCache](/cacher/propertyvalueattributecache): + +```csharp +var attribute = PropertyValueAttributeCache + .Get(e => e.Name)? + .FirstOrDefault(e => e.GetType() == typeof(SourceColumnNullMappingAttribute)); +``` + +{: .important } +> We strongly recommend using [PropertyValueAttributeCache](/cacher/propertyvalueattributecache) for maximum performance. diff --git a/pages/attributes/vertica/sourceversion.md b/pages/attributes/vertica/sourceversion.md new file mode 100644 index 0000000..cf6f4a3 --- /dev/null +++ b/pages/attributes/vertica/sourceversion.md @@ -0,0 +1,51 @@ +--- +layout: default +title: SourceVersion +permalink: /attribute/vertica/sourceversion +tags: [repodb, attribute, sourceversion] +parent: "Vertica" +grand_parent: ATTRIBUTES +--- + +# SourceVersion + +--- + +This attribute sets the `VerticaParameter.SourceVersion` property value via a class property. + +### Attribute + +Example usage: + +```csharp +public class Person +{ + public int Id { get; set; } + + [SourceVersion(DataRowVersion.Current)] + public string Name { get; set; } +} +``` + +### Fluent Mapping + +To configure via [FluentMapper](/mapper/fluentmapper): + +```csharp +FluentMapper + .Entity() + .PropertyValueAttributes(e => e.Name, new SourceVersionAttribute(DataRowVersion.Current)); +``` + +### Retrieval + +Retrieve the attribute via [PropertyValueAttributeCache](/cacher/propertyvalueattributecache): + +```csharp +var attribute = PropertyValueAttributeCache + .Get(e => e.Name)? + .FirstOrDefault(e => e.GetType() == typeof(SourceVersionAttribute)); +``` + +{: .important } +> We strongly recommend using [PropertyValueAttributeCache](/cacher/propertyvalueattributecache) for maximum performance. diff --git a/pages/attributes/vertica/vertica.md b/pages/attributes/vertica/vertica.md new file mode 100644 index 0000000..653cf6b --- /dev/null +++ b/pages/attributes/vertica/vertica.md @@ -0,0 +1,13 @@ +--- +layout: default +title: "Vertica" +has_children: true +permalink: /attribute/vertica +parent: ATTRIBUTES +--- + +# Attributes +{: .fs-9 } + +Attributes for decorating VerticaParameter objects. +{: .fs-6 .fw-300 } diff --git a/pages/attributes/vertica/verticatype.md b/pages/attributes/vertica/verticatype.md new file mode 100644 index 0000000..f809dcb --- /dev/null +++ b/pages/attributes/vertica/verticatype.md @@ -0,0 +1,51 @@ +--- +layout: default +title: VerticaType +permalink: /attribute/vertica/verticatype +tags: [repodb, attribute, verticatype] +parent: "Vertica" +grand_parent: ATTRIBUTES +--- + +# VerticaType + +--- + +This attribute sets the `VerticaParameter.Type` property value via a class property. + +### Attribute + +Example usage: + +```csharp +public class Person +{ + public int Id { get; set; } + + [VerticaType(VerticaType.VarChar)] + public string Name { get; set; } +} +``` + +### Fluent Mapping + +To configure via [FluentMapper](/mapper/fluentmapper): + +```csharp +FluentMapper + .Entity() + .PropertyValueAttributes(e => e.Name, new VerticaTypeAttribute(VerticaType.VarChar)); +``` + +### Retrieval + +Retrieve the attribute via [PropertyValueAttributeCache](/cacher/propertyvalueattributecache): + +```csharp +var attribute = PropertyValueAttributeCache + .Get(e => e.Name)? + .FirstOrDefault(e => e.GetType() == typeof(VerticaTypeAttribute)); +``` + +{: .important } +> We strongly recommend using [PropertyValueAttributeCache](/cacher/propertyvalueattributecache) for maximum performance. diff --git a/pages/classes/vertica/dbtypenametocolumnnameresolver.md b/pages/classes/vertica/dbtypenametocolumnnameresolver.md new file mode 100644 index 0000000..b367826 --- /dev/null +++ b/pages/classes/vertica/dbtypenametocolumnnameresolver.md @@ -0,0 +1,31 @@ +--- +layout: default +sidebar: classes +title: "DbTypeNameToColumnNameResolver" +description: "A class used to resolve a Vertica database type name into its equivalent base Vertica column type keyword." +permalink: /class/vertica/dbtypenametocolumnnameresolver +tags: [repodb, dbtypenametocolumnnameresolver, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# DbTypeNameToColumnNameResolver + +--- + +This [IResolver](/interface/iresolver)`` implementation converts a Vertica database type name — e.g. a [DbField](/class/dbfield)'s `DatabaseType` — into its equivalent *base* Vertica column type keyword (e.g. `numeric` → `NUMERIC`, `varchar` → `VARCHAR`). Sized types (`numeric`, `decimal`, `char`, `varchar`, `binary`, `varbinary`) are returned without their `(precision,scale)`/`(size)` portion — the caller is expected to append that using the field's own precision/scale/size. An unrecognized database type falls back to `LONG VARCHAR`, Vertica's large-text type. + +It is used internally by [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) to generate the column definitions of the pseudo (staging) table backing `BulkMerge`, `BulkUpdate`, `BulkDelete`, `BulkDeleteByKey`, and `BulkInsert` with [VerticaBulkImportIdentityBehavior.ReturnIdentity](/enumeration/vertica/verticabulkimportidentitybehavior). + +{: .note } +> This class shares its unqualified name with Firebird's own [DbTypeNameToColumnNameResolver](/class/firebird/dbtypenametocolumnnameresolver) — the two live in separate assemblies/namespaced provider folders, so there is no compile-time conflict, but the same name resolves differently depending on which provider package is referenced. + +## Usability + +```csharp +var resolver = new DbTypeNameToColumnNameResolver(); +var baseType = resolver.Resolve("decimal"); // "DECIMAL" + +// The caller appends sizing itself, e.g.: +var columnType = $"{baseType}({field.Precision ?? 18},{field.Scale ?? 0})"; // "DECIMAL(18,2)" +``` diff --git a/pages/classes/vertica/dbtypetoverticastringnameresolver.md b/pages/classes/vertica/dbtypetoverticastringnameresolver.md new file mode 100644 index 0000000..633b9c2 --- /dev/null +++ b/pages/classes/vertica/dbtypetoverticastringnameresolver.md @@ -0,0 +1,23 @@ +--- +layout: default +sidebar: classes +title: "DbTypeToVerticaStringNameResolver" +description: "A class used to resolve a DbType into its equivalent Vertica database string name." +permalink: /class/vertica/dbtypetoverticastringnameresolver +tags: [repodb, dbtypetoverticastringnameresolver, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# DbTypeToVerticaStringNameResolver + +--- + +This [IResolver](/interface/iresolver)`` implementation converts a .NET `DbType` into its equivalent Vertica SQL type name (e.g. `DbType.String` → `VARCHAR(8191)`, `DbType.Guid` → `UUID`, `DbType.Binary` → `VARBINARY(65000)`). It is used internally by [VerticaConvertFieldResolver](/class/vertica/verticaconvertfieldresolver) to build `CAST(...)` expressions. + +## Usability + +```csharp +var resolver = new DbTypeToVerticaStringNameResolver(); +var typeName = resolver.Resolve(DbType.Int64); // "BIGINT" +``` diff --git a/pages/classes/vertica/timetodatetimepropertyhandler.md b/pages/classes/vertica/timetodatetimepropertyhandler.md new file mode 100644 index 0000000..dfc6d9a --- /dev/null +++ b/pages/classes/vertica/timetodatetimepropertyhandler.md @@ -0,0 +1,28 @@ +--- +layout: default +sidebar: classes +title: "TimeToDateTimePropertyHandler" +description: "A property handler that re-bases the date component of a value read back from a Vertica TIME column." +permalink: /class/vertica/timetodatetimepropertyhandler +tags: [repodb, timetodatetimepropertyhandler, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# TimeToDateTimePropertyHandler + +--- + +Vertica's driver returns a `TIME` column's value combined with today's date rather than a fixed placeholder date. This [IPropertyHandler](/interface/ipropertyhandler) re-bases the date component of a value read back from a `TIME` column onto `DateTime`'s default (`0001-01-01`) date, keeping only its time-of-day. `Set` passes the value through unchanged — Vertica only stores the time-of-day portion of a bound value in a `TIME` column regardless of its date component. + +{: .important } +> This class lives in the `RepoDb.PropertyHandlers.Vertica` namespace. A second, near-identical class named `VerticaTimeToDateTimePropertyHandler` also exists in the plain `RepoDb.PropertyHandlers` namespace — it is not referenced anywhere in the library or its test suite and appears to be leftover duplicate code from a refactor. Prefer `TimeToDateTimePropertyHandler` (this class), which is the one the library's own integration tests register. + +## Usability + +Register it explicitly, scoped to the specific entity property that maps to a `TIME` column. + +```csharp +PropertyHandlerMapper.Add( + e => e.StartTime, new RepoDb.PropertyHandlers.Vertica.TimeToDateTimePropertyHandler(), true); +``` diff --git a/pages/classes/vertica/vertica.md b/pages/classes/vertica/vertica.md new file mode 100644 index 0000000..ef6ed62 --- /dev/null +++ b/pages/classes/vertica/vertica.md @@ -0,0 +1,15 @@ +--- +layout: default +title: "Vertica" +has_children: true +permalink: /class/vertica +parent: CLASSES +--- + +# Classes (Vertica) + +--- + +Classes specific to the Vertica data provider ([RepoDb.Vertica](https://www.nuget.org/packages/RepoDb.Vertica), built on [Vertica.Data](https://www.nuget.org/packages/Vertica.Data)). These cover connection bootstrapping and configuration ([VerticaBootstrap](/class/vertica/verticabootstrap), [VerticaConfiguration](/class/vertica/verticaconfiguration)), schema discovery ([VerticaDbHelper](/class/vertica/verticadbhelper)), SQL generation ([VerticaStatementBuilder](/class/vertica/verticastatementbuilder)), provider settings ([VerticaDbSetting](/class/vertica/verticadbsetting)), type resolution between .NET, `DbType` and Vertica types ([DbTypeToVerticaStringNameResolver](/class/vertica/dbtypetoverticastringnameresolver), [VerticaDbTypeNameToClientTypeResolver](/class/vertica/verticadbtypenametoclienttyperesolver), [VerticaConvertFieldResolver](/class/vertica/verticaconvertfieldresolver), [DbTypeNameToColumnNameResolver](/class/vertica/dbtypenametocolumnnameresolver)), and a property handler for a CLR type mismatch with no native Vertica equivalent ([TimeToDateTimePropertyHandler](/class/vertica/timetodatetimepropertyhandler)). + +Also included are the support classes used by the [Vertica bulk operations](/operation/vertica) ([VerticaBulkInsertMapItem](/class/vertica/verticabulkinsertmapitem), [VerticaTraceKeys](/class/vertica/verticatracekeys)), part of the separate [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) package. diff --git a/pages/classes/vertica/verticabootstrap.md b/pages/classes/vertica/verticabootstrap.md new file mode 100644 index 0000000..b33893d --- /dev/null +++ b/pages/classes/vertica/verticabootstrap.md @@ -0,0 +1,38 @@ +--- +layout: default +sidebar: classes +title: "VerticaBootstrap" +description: "A class that is being used to initialize the necessary settings for the VerticaConnection object." +permalink: /class/vertica/verticabootstrap +tags: [repodb, verticabootstrap, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaBootstrap + +--- + +This class initializes the necessary dependencies for the `VerticaConnection` object — the [DbSetting](/class/vertica/verticadbsetting), [DbHelper](/class/vertica/verticadbhelper) and [StatementBuilder](/class/vertica/verticastatementbuilder) — and registers them via their respective mappers. + +## Properties + +| Name | Description | +|:-----|:------------| +| IsInitialized | Returns `true` once the initialization has completed. | + +## Usability + +Call [VerticaConfiguration.UseVertica()](/class/vertica/verticaconfiguration) during application start-up; it triggers this class internally. + +```csharp +GlobalConfiguration + .Setup() + .UseVertica(); +``` + +{: .note } +> Initialization is a one-time, idempotent operation — calling `UseVertica()` more than once has no additional effect. + +{: .important } +> Initialization also forces `CultureInfo.CurrentCulture` to `CultureInfo.InvariantCulture` for the calling thread, and for every subsequently-created thread in the process, working around `Vertica.Data` formatting date-like parameter values using the ambient thread culture instead of the invariant one. diff --git a/pages/classes/vertica/verticabulkinsertmapitem.md b/pages/classes/vertica/verticabulkinsertmapitem.md new file mode 100644 index 0000000..de044e4 --- /dev/null +++ b/pages/classes/vertica/verticabulkinsertmapitem.md @@ -0,0 +1,49 @@ +--- +layout: default +sidebar: classes +title: "VerticaBulkInsertMapItem" +description: "A mapping class used to define a column mapping, with an optional explicit VerticaType, for the Vertica bulk operations." +permalink: /class/vertica/verticabulkinsertmapitem +tags: [repodb, verticabulkinsertmapitem, vertica, bulk] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaBulkInsertMapItem + +--- + +This class extends [BulkInsertMapItem](/class/bulkinsertmapitem) with an optional, explicit `VerticaType` to bind with for the mapped column. It is not currently consumed by the `COPY`-stream-based bulk-copy implementation — Vertica's `COPY` parser infers each field's wire format from the destination column's actual server-side type — so it is kept only as a forward-looking escape hatch, matching the equivalent parameter on every other bulk-operations package's map-item type. + +Used by the Vertica [BulkInsert](/operation/vertica/bulkinsert), [BulkMerge](/operation/vertica/bulkmerge) and [BulkUpdate](/operation/vertica/bulkupdate) operations. + +## Create a new Instance + +```csharp +var mapItem = new VerticaBulkInsertMapItem("SourceId", "DestinationId"); +``` + +Or with an explicit `VerticaType`: + +```csharp +var mapItem = new VerticaBulkInsertMapItem("SourceName", "DestinationName", VerticaType.VarChar); +``` + +## Usage for BulkOperations + +```csharp +var mappings = new [] +{ + new VerticaBulkInsertMapItem("FirstName", "FName"), + new VerticaBulkInsertMapItem("LastName", "LName") +}; + +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(100000); + connection.BulkInsert(people, mappings: mappings); +} +``` + +{: .note } +> The same approach applies to [BulkMerge](/operation/vertica/bulkmerge) and [BulkUpdate](/operation/vertica/bulkupdate). The `mappings` argument is optional — omitting it causes the library to auto-map columns by name (case-insensitive). diff --git a/pages/classes/vertica/verticaconfiguration.md b/pages/classes/vertica/verticaconfiguration.md new file mode 100644 index 0000000..30355b1 --- /dev/null +++ b/pages/classes/vertica/verticaconfiguration.md @@ -0,0 +1,27 @@ +--- +layout: default +sidebar: classes +title: "VerticaConfiguration" +description: "A class that is being used to initialize the necessary settings for the Vertica data provider." +permalink: /class/vertica/verticaconfiguration +tags: [repodb, verticaconfiguration, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaConfiguration + +--- + +This class exposes the `UseVertica()` extension method of [GlobalConfiguration](/class/globalconfiguration), which wires up all the necessary dependencies for Vertica (via [VerticaBootstrap](/class/vertica/verticabootstrap)). + +## Usability + +```csharp +GlobalConfiguration + .Setup() + .UseVertica(); +``` + +{: .note } +> Call this once during application start-up, before opening any `VerticaConnection`. diff --git a/pages/classes/vertica/verticaconvertfieldresolver.md b/pages/classes/vertica/verticaconvertfieldresolver.md new file mode 100644 index 0000000..7701740 --- /dev/null +++ b/pages/classes/vertica/verticaconvertfieldresolver.md @@ -0,0 +1,24 @@ +--- +layout: default +sidebar: classes +title: "VerticaConvertFieldResolver" +description: "A class used to resolve the Field name conversion for Vertica." +permalink: /class/vertica/verticaconvertfieldresolver +tags: [repodb, verticaconvertfieldresolver, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaConvertFieldResolver + +--- + +This class resolves a [Field](/class/field) into a `CAST(column AS TYPE)` SQL fragment whenever the field carries an explicit `.Type`, using [DbTypeToVerticaStringNameResolver](/class/vertica/dbtypetoverticastringnameresolver) to determine the Vertica type name. It is used internally by [VerticaStatementBuilder](/class/vertica/verticastatementbuilder) and is not typically used directly. + +## Usability + +```csharp +var resolver = new VerticaConvertFieldResolver(); +var expression = resolver.Resolve(new Field("Age", typeof(int)), dbSetting); +// CAST("Age" AS INTEGER) +``` diff --git a/pages/classes/vertica/verticadbhelper.md b/pages/classes/vertica/verticadbhelper.md new file mode 100644 index 0000000..bb35ddd --- /dev/null +++ b/pages/classes/vertica/verticadbhelper.md @@ -0,0 +1,40 @@ +--- +layout: default +sidebar: classes +title: "VerticaDbHelper" +description: "A helper class that is being used to retrieve the schema information (columns, primary/identity key) of a Vertica table." +permalink: /class/vertica/verticadbhelper +tags: [repodb, verticadbhelper, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaDbHelper + +--- + +This class implements [IDbHelper](/interface/idbhelper) for Vertica. It queries Vertica's own `v_catalog.columns`/`v_catalog.primary_keys` system tables to build the list of [DbField](/class/dbfield) objects RepoDB uses internally to generate SQL statements, stripping the `(size)`/`(precision,scale)` suffix off each column's raw `data_type` value down to its base type-name keyword before handing it to [VerticaDbTypeNameToClientTypeResolver](/class/vertica/verticadbtypenametoclienttyperesolver). + +It is automatically registered by [VerticaBootstrap](/class/vertica/verticabootstrap) — you do not need to instantiate it directly under normal use. + +## Properties + +| Name | Description | +|:-----|:------------| +| DbTypeResolver | The `IResolver` used to convert a Vertica column type name into its equivalent .NET CLR type. Defaults to [VerticaDbTypeNameToClientTypeResolver](/class/vertica/verticadbtypenametoclienttyperesolver). | + +## GetScopeIdentity + +`GetScopeIdentity`/`GetScopeIdentityAsync` execute `SELECT LAST_INSERT_ID()` against the connection. + +{: .important } +> This has not been verified against a live Vertica instance. Verify that `LAST_INSERT_ID()` is genuinely supported by your Vertica version/session before relying on it in production. + +## Usability + +Only override this if you need a custom type resolver. + +```csharp +var dbHelper = new VerticaDbHelper(new MyCustomVerticaDbTypeNameToClientTypeResolver()); +DbHelperMapper.Add(dbHelper, true); +``` diff --git a/pages/classes/vertica/verticadbsetting.md b/pages/classes/vertica/verticadbsetting.md new file mode 100644 index 0000000..83e1b36 --- /dev/null +++ b/pages/classes/vertica/verticadbsetting.md @@ -0,0 +1,56 @@ +--- +layout: default +sidebar: classes +title: "VerticaDbSetting" +description: "A setting class used for the Vertica data provider." +permalink: /class/vertica/verticadbsetting +tags: [repodb, verticadbsetting, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaDbSetting + +--- + +This class is the [BaseDbSetting](/class/basedbsetting)-derived implementation for Vertica. It is automatically registered by [VerticaBootstrap](/class/vertica/verticabootstrap) — you do not need to instantiate it directly under normal use. + +## Configured Values + +| Property | Value | +|:---------|:------| +| AreTableHintsSupported | `false` | +| ClosingQuote | `"` | +| DefaultSchema | `null` | +| IsAffectedRowsSupported | `true` | +| IsDirectionSupported | `false` | +| IsExecuteReaderDisposable | `false` | +| IsInsertAllBatchable | `true` | +| IsMultiStatementExecutable | `false` | +| IsPreparable | `true` | +| IsTransactionSupported | `true` | +| IsUseUpsert | `true` | +| MaxParameterCount | `1500` | +| MultiStatementSeparator | `;` | +| OpeningQuote | `"` | +| ParameterPrefix | `@` | +| RequiresDbTypeBeforeValue | `true` | +| SkipsUnreferencedParameters | `true` | +| SqlTextParameterPrefix | `@` | + +{: .note } +> `IsMultiStatementExecutable` is `false` — `VerticaCommand` refuses a compound `;`-separated statement once it carries a parameter — yet `IsInsertAllBatchable` is `true`, so [InsertAll](/operation/insertall) still batches multiple rows into one genuine multi-row `INSERT ... VALUES (...), (...), ...` statement. [MergeAll](/operation/mergeall)/[UpdateAll](/operation/updateall) have no equivalent single-statement shape and issue one round trip per row; passing an explicit `batchSize` greater than `1` to either throws a `NotSupportedException`. + +{: .note } +> `RequiresDbTypeBeforeValue` is `true` — `Vertica.Data` lazily initializes internal parameter state inside `VerticaParameter.Type`'s setter and throws if `Value` is assigned first on a parameter fresh off `VerticaCommand.CreateParameter()`. + +{: .note } +> `SkipsUnreferencedParameters` is `true` — Vertica strictly validates that every parameter bound to a command is actually referenced by a placeholder in the command text, rejecting the whole command otherwise (e.g. a null-valued equality filter rendered as a literal `IS NULL` with no placeholder at all). + +## Usability + +Use [DbSettingMapper](/mapper/dbsettingmapper) to override it with a custom implementation. + +```csharp +DbSettingMapper.Add(typeof(VerticaConnection), new MyCustomVerticaDbSetting(), true); +``` diff --git a/pages/classes/vertica/verticadbtypenametoclienttyperesolver.md b/pages/classes/vertica/verticadbtypenametoclienttyperesolver.md new file mode 100644 index 0000000..6bc76c1 --- /dev/null +++ b/pages/classes/vertica/verticadbtypenametoclienttyperesolver.md @@ -0,0 +1,26 @@ +--- +layout: default +sidebar: classes +title: "VerticaDbTypeNameToClientTypeResolver" +description: "A class used to resolve a Vertica database type name into its equivalent .NET CLR type." +permalink: /class/vertica/verticadbtypenametoclienttyperesolver +tags: [repodb, verticadbtypenametoclienttyperesolver, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaDbTypeNameToClientTypeResolver + +--- + +This [IResolver](/interface/iresolver)`` implementation converts a Vertica column type name — as returned by `v_catalog.columns.data_type`, with its `(size)`/`(precision,scale)` suffix already stripped — into its equivalent .NET CLR type. It is the default `DbTypeResolver` used by [VerticaDbHelper](/class/vertica/verticadbhelper). + +{: .note } +> Verified directly against `VerticaDataReader.GetSchemaTable()`: Vertica has no distinct storage widths for its integer or floating-point types — `SMALLINT`/`INTEGER`/`BIGINT`/etc. are all synonyms for one 8-byte integer (reported as `int`), and `FLOAT`/`DOUBLE PRECISION`/`REAL` are all synonyms for one 8-byte float (reported as `float`) — so both resolve to their widest CLR type (`long`/`double`), not `int`/`float`. `TIME` is reported back as `System.DateTime`, not `TimeSpan`. + +## Usability + +```csharp +var resolver = new VerticaDbTypeNameToClientTypeResolver(); +var clrType = resolver.Resolve("varchar"); // typeof(string) +``` diff --git a/pages/classes/vertica/verticastatementbuilder.md b/pages/classes/vertica/verticastatementbuilder.md new file mode 100644 index 0000000..ce5148b --- /dev/null +++ b/pages/classes/vertica/verticastatementbuilder.md @@ -0,0 +1,47 @@ +--- +layout: default +sidebar: classes +title: "VerticaStatementBuilder" +description: "A class used to build the SQL statements for Vertica." +permalink: /class/vertica/verticastatementbuilder +tags: [repodb, verticastatementbuilder, vertica] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaStatementBuilder + +--- + +This class is the [BaseStatementBuilder](/class/basestatementbuilder)-derived implementation for Vertica. It is automatically registered by [VerticaBootstrap](/class/vertica/verticabootstrap) — you do not need to instantiate it directly under normal use. + +## Constructors + +```csharp +public VerticaStatementBuilder() +public VerticaStatementBuilder(IDbSetting dbSetting, + IResolver convertFieldResolver = null, + IResolver averageableClientTypeResolver = null) +``` + +`convertFieldResolver` defaults to [VerticaConvertFieldResolver](/class/vertica/verticaconvertfieldresolver), used to render `CAST(...)` expressions for typed fields. + +## Vertica-specific generation notes + +- Paging uses `LIMIT`/`LIMIT ... OFFSET` for [Query](/operation/query)/top-N queries and [BatchQuery](/operation/batchquery)/skip-take queries — no `TOP`/`FIRST` keyword needed. +- [InsertAll](/operation/insertall) generates a genuine multi-row `INSERT INTO ... VALUES (...), (...), ...` statement, since `VerticaDbSetting.IsInsertAllBatchable` is `true` even though `IsMultiStatementExecutable` is `false`. +- [Merge](/operation/merge)/[MergeAll](/operation/mergeall) never emit a native `MERGE` statement at all — Vertica rejects `MERGE` outright against any table with an `IDENTITY`/`AUTO_INCREMENT` column, and has no procedural fallback equivalent to Firebird's `EXECUTE BLOCK`. Instead, an `UPDATE ... WHERE qualifiers` is joined with a trailing `; INSERT ... WHERE NOT EXISTS (...)` into one command text, with a follow-up `SELECT LAST_INSERT_ID()` (or a `CASE WHEN IS NULL THEN LAST_INSERT_ID() ELSE END`, when the identity column is itself a qualifier) appended when a key needs to be returned. +- [Truncate](/operation/truncate) compiles to a plain `DELETE FROM t` — Vertica has no `TRUNCATE TABLE` statement (as of 5.0) — which does not reset an `IDENTITY` column's next value. +- Every generated statement omits the trailing `;` that [BaseStatementBuilder](/class/basestatementbuilder) normally appends, since Vertica's DSQL layer rejects a trailing statement terminator on a statement submitted through `VerticaCommand.CommandText`. +- Passing a non-null `hints` argument to any `Create*` method throws `NotSupportedException`, since `VerticaDbSetting.AreTableHintsSupported` is `false`. + +{: .important } +> [Merge](/operation/merge)/[MergeAll](/operation/mergeall)'s `UPDATE ...; INSERT ...` command text is a compound, `;`-joined statement carrying parameters in both halves — this has not been verified against a live Vertica instance, and conflicts with the same compound-statement restriction documented elsewhere for `VerticaCommand` (see [Operations (Vertica)](/operation/vertica)). Verify this end-to-end before relying on [Merge](/operation/merge)/[MergeAll](/operation/mergeall) in production. + +## Usability + +Use [StatementBuilderMapper](/mapper/statementbuildermapper) to override it with a custom implementation. + +```csharp +StatementBuilderMapper.Add(typeof(VerticaConnection), new MyCustomVerticaStatementBuilder(dbSetting), true); +``` diff --git a/pages/classes/vertica/verticatracekeys.md b/pages/classes/vertica/verticatracekeys.md new file mode 100644 index 0000000..feb5d04 --- /dev/null +++ b/pages/classes/vertica/verticatracekeys.md @@ -0,0 +1,39 @@ +--- +layout: default +sidebar: classes +title: "VerticaTraceKeys" +description: "A class that holds the constant values of the operation tracing keys used by the Vertica bulk operations." +permalink: /class/vertica/verticatracekeys +tags: [repodb, verticatracekeys, vertica, bulk] +parent: "Vertica" +grand_parent: CLASSES +--- + +# VerticaTraceKeys + +--- + +This class holds the tracing key constants used by the [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) bulk operations, for use with [ITrace](/interface/itrace). + +## Fields + +| Name | Value | +|:-----|:------| +| VerticaBulkDelete | `"VerticaBulkDelete"` | +| VerticaBulkDeleteByKey | `"VerticaBulkDeleteByKey"` | +| VerticaBulkInsert | `"VerticaBulkInsert"` | +| VerticaBulkMerge | `"VerticaBulkMerge"` | +| VerticaBulkUpdate | `"VerticaBulkUpdate"` | + +## Usability + +Pass a custom `traceKey` value, or compare against these constants inside a custom [ITrace](/interface/itrace) implementation. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var insertedRows = connection.BulkInsert(people, + trace: new MyCustomTrace(), + traceKey: VerticaTraceKeys.VerticaBulkInsert); +} +``` diff --git a/pages/enumerations/vertica/vertica.md b/pages/enumerations/vertica/vertica.md new file mode 100644 index 0000000..3a6780b --- /dev/null +++ b/pages/enumerations/vertica/vertica.md @@ -0,0 +1,13 @@ +--- +layout: default +title: "Vertica" +has_children: true +permalink: /enumeration/vertica +parent: ENUMERATIONS +--- + +# Enumerations (Vertica) + +--- + +Enumerations used by the [Vertica bulk operations](/operation/vertica). [VerticaBulkImportIdentityBehavior](/enumeration/vertica/verticabulkimportidentitybehavior) controls whether newly generated identity values are returned back to the entities after [BulkInsert](/operation/vertica/bulkinsert) or [BulkMerge](/operation/vertica/bulkmerge), and [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype) controls the kind of staging table created to stage the bulk-imported data. diff --git a/pages/enumerations/vertica/verticabulkimportidentitybehavior.md b/pages/enumerations/vertica/verticabulkimportidentitybehavior.md new file mode 100644 index 0000000..67a49d6 --- /dev/null +++ b/pages/enumerations/vertica/verticabulkimportidentitybehavior.md @@ -0,0 +1,52 @@ +--- +layout: default +sidebar: enumerations +title: "VerticaBulkImportIdentityBehavior" +description: "An enumeration that is being used to define the behavior of the identity property/column when an entity is being bulk-imported towards the target table." +permalink: /enumeration/vertica/verticabulkimportidentitybehavior +tags: [repodb, verticabulkimportidentitybehavior] +parent: "Vertica" +grand_parent: ENUMERATIONS +--- + +# VerticaBulkImportIdentityBehavior + +--- + +This enum defines the behavior of the identity property/column when an entity is bulk-imported into a target table. It applies only to [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). + +## Enum Values + +| Name | Description | +|:-----|:------------| +| KeepIdentity | The value of the identity property/column will be kept and used. (This is the default value) | +| ReturnIdentity | The newly generated identity value from the target table will be set back to the entity. | + +{: .note } +> There is no `Unspecified` state — this enum defaults straight to `KeepIdentity`, the same shape as [Db2BulkImportIdentityBehavior](/enumeration/db2/db2bulkimportidentitybehavior) and [FirebirdBulkImportIdentityBehavior](/enumeration/firebird/firebirdbulkimportidentitybehavior). + +## Usability + +This enum is used by both the [BulkInsert](/operation/vertica/bulkinsert) and [BulkMerge](/operation/vertica/bulkmerge) operations of [RepoDb.Vertica.BulkOperations](/operation/vertica). Pass the value to the `identityBehavior` argument when calling the operation. + +For [BulkInsert](/operation/vertica/bulkinsert): + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(1000); + var insertedRows = connection.BulkInsert(people, + identityBehavior: VerticaBulkImportIdentityBehavior.ReturnIdentity); +} +``` + +For [BulkMerge](/operation/vertica/bulkmerge): + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(1000); + var mergedRows = connection.BulkMerge(people, + identityBehavior: VerticaBulkImportIdentityBehavior.ReturnIdentity); +} +``` diff --git a/pages/enumerations/vertica/verticabulkimportpseudotabletype.md b/pages/enumerations/vertica/verticabulkimportpseudotabletype.md new file mode 100644 index 0000000..25537b4 --- /dev/null +++ b/pages/enumerations/vertica/verticabulkimportpseudotabletype.md @@ -0,0 +1,49 @@ +--- +layout: default +sidebar: enumerations +title: "VerticaBulkImportPseudoTableType" +description: "An enumeration that is being used to define the type of staging (pseudo) table to be created during the bulk-import operations." +permalink: /enumeration/vertica/verticabulkimportpseudotabletype +tags: [repodb, verticabulkimportpseudotabletype] +parent: "Vertica" +grand_parent: ENUMERATIONS +--- + +# VerticaBulkImportPseudoTableType + +--- + +This enum defines the type of staging (pseudo) table created during bulk-import operations. It applies only to [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). + +It is used by the following bulk operations, all part of [RepoDb.Vertica.BulkOperations](/operation/vertica). + +- [BulkDelete](/operation/vertica/bulkdelete) +- [BulkDeleteByKey](/operation/vertica/bulkdeletebykey) +- [BulkInsert](/operation/vertica/bulkinsert) (only when `identityBehavior` is `ReturnIdentity`) +- [BulkMerge](/operation/vertica/bulkmerge) +- [BulkUpdate](/operation/vertica/bulkupdate) + +## Enum Values + +| Name | Description | +|:-----|:------------| +| Auto | Chooses between `Physical` and `Memory` based on row count (`Physical` at 5,000 rows or more). This is the default. | +| Memory | Backs the operation with a Vertica `GLOBAL TEMPORARY TABLE ... ON COMMIT PRESERVE ROWS`. Rows are private to the connection that wrote them. | +| Physical | Backs the operation with an ordinary heap table. Faster to create for very large row counts than a global temporary table's per-connection storage, at the cost of the rows briefly existing as an ordinary (if uniquely-named) table. | + +{: .note } +> Every pseudo table is created with a per-call unique name, so unlike some other providers' bulk-operations packages, `Physical` and `Memory` are both safe for concurrent callers writing against the same target table — there is no shared, deterministic staging-table name for them to race on. + +## Usability + +Pass the value to the `pseudoTableType` argument of the target operation. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(1000); + var insertedRows = connection.BulkInsert(people, + identityBehavior: VerticaBulkImportIdentityBehavior.ReturnIdentity, + pseudoTableType: VerticaBulkImportPseudoTableType.Physical); +} +``` diff --git a/pages/features/bulkoperations.md b/pages/features/bulkoperations.md index 245a680..5b401b6 100644 --- a/pages/features/bulkoperations.md +++ b/pages/features/bulkoperations.md @@ -29,6 +29,7 @@ Bulk operations are available for the following providers, each via its own exte - [PostgreSQL](/operation/postgresql) — via [RepoDb.PostgreSql.BulkOperations](https://www.nuget.org/packages/RepoDb.PostgreSql.BulkOperations), built on Npgsql's [NpgsqlBinaryImporter](https://www.npgsql.org/doc/api/Npgsql.NpgsqlBinaryImporter.html) (exposed as the `BinaryBulk*` methods). - [Db2](/operation/db2) — via [RepoDb.Db2.BulkOperations](https://www.nuget.org/packages/RepoDb.Db2.BulkOperations), built on the IBM Data Server .NET Provider's [DB2BulkCopy](https://www.ibm.com/docs/en/db2/11.5?topic=classes-db2bulkcopy-class). - [Firebird](/operation/firebird) — via [RepoDb.Firebird.BulkOperations](https://www.nuget.org/packages/RepoDb.Firebird.BulkOperations), built on `FbBatchCommand`, the FirebirdSql.Data.FirebirdClient driver's native ADO.NET batching API. +- [Vertica](/operation/vertica) — via [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations), built on `VerticaCopyStream`, `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. Each provider page linked above documents its own underlying mechanics, generated SQL statements, special arguments, and identity-setting alignment, since the implementation differs per ADO.NET driver. diff --git a/pages/getstarted/vertica.md b/pages/getstarted/vertica.md new file mode 100644 index 0000000..15073e1 --- /dev/null +++ b/pages/getstarted/vertica.md @@ -0,0 +1,307 @@ +--- +layout: default +sidebar: getstarted +title: "Vertica" +description: "Learn on how to work with Vertica databases using RepoDB library." +permalink: /tutorial/get-started-vertica +tags: [repodb, tutorial, get-started, orm, hybrid-orm, vertica] +parent: GET STARTED +--- + +# Get Started for Vertica + +--- + +RepoDB is a hybrid .NET ORM library for [Vertica](https://www.nuget.org/packages/RepoDb.Vertica) Database. The project is hosted at [Github](https://github.com/mikependon/RepoDb/tree/master/RepoDb.Vertica) and is licensed with [Apache 2.0](http://apache.org/licenses/LICENSE-2.0.html). + +Support ships as two packages: + +- [RepoDb.Vertica](https://www.nuget.org/packages/RepoDb.Vertica) — the core provider, built on [Vertica.Data](https://www.nuget.org/packages/Vertica.Data). +- [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) — adds `BulkInsert`, `BulkMerge`, `BulkUpdate`, `BulkDelete` and `BulkDeleteByKey`. + +## Installation + +Install the library via NuGet using the Package Manager Console. + +```csharp +> Install-Package RepoDb.Vertica +``` + +After installation, call the globalized setup method to initialize all dependencies for Vertica. + +```csharp +GlobalConfiguration + .Setup() + .UseVertica(); +``` + +{: .important } +> `UseVertica()` also forces the calling thread's, and every subsequently-created thread's, `CultureInfo.CurrentCulture` to `CultureInfo.InvariantCulture`. `Vertica.Data` formats/re-parses date-like parameter values using the ambient thread culture rather than `CultureInfo.InvariantCulture` — on a machine whose culture renders time with a non-colon separator (e.g. `en-DK`'s `13.45.30`), this corrupts the value the driver actually sends. There is no per-call interception point available to a provider, so this is applied process-wide rather than scoped to Vertica calls specifically. + +To use bulk operations (`BulkDelete`, `BulkDeleteByKey`, `BulkInsert`, `BulkMerge` and `BulkUpdate` — see [Operations (Vertica)](/operation/vertica)), install the [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) package. + +```csharp +> Install-Package RepoDb.Vertica.BulkOperations +``` + +## Create a Table + +The examples below assume the following table exists in the database. + +```csharp +CREATE TABLE "Person" +( + "Id" IDENTITY(1, 1), + "Name" VARCHAR(128), + "Age" INTEGER, + "CreatedDateUtc" TIMESTAMP +); +``` + +## Create a Model + +The examples below assume the following model exists in the application. + +```csharp +public class Person +{ + public long Id { get; set; } + public string Name { get; set; } + public long Age { get; set; } + public DateTime CreatedDateUtc { get; set; } +} +``` + +{: .note } +> Vertica has no distinct storage widths for its integer types — `SMALLINT`/`INTEGER`/`BIGINT` are all synonyms for one 8-byte integer, reported back to ADO.NET as `long`. Map integer-looking columns as `long` rather than `int` to avoid a cast failure when reading them back. + +## Creating a Record + +To insert a row, use the [Insert](/operation/insert) method. + +```csharp +var person = new Person +{ + Name = "John Doe", + Age = 54, + CreatedDateUtc = DateTime.UtcNow +}; +using (var connection = new VerticaConnection(ConnectionString)) +{ + var id = connection.Insert(person); +} +``` + +To insert multiple rows, use the [InsertAll](/operation/insertall) operation. + +```csharp +var people = GetPeople(100); +using (var connection = new VerticaConnection(ConnectionString)) +{ + var rowsInserted = connection.InsertAll(people); +} +``` + +{: .note } +> Unlike most providers, Vertica's `IsMultiStatementExecutable` is `false` (`VerticaCommand` refuses a compound `;`-separated statement once it carries a parameter) yet [InsertAll](/operation/insertall) still batches multiple rows into one genuine multi-row `INSERT ... VALUES (...), (...), ...` statement — the `IsInsertAllBatchable` database setting overrides `IsMultiStatementExecutable` specifically for this shape. [MergeAll](/operation/mergeall)/[UpdateAll](/operation/updateall), which have no equivalent single-statement shape, still issue one round trip per row; passing an explicit `batchSize` greater than `1` to either throws a `NotSupportedException`. + +## Querying a Record + +To query a row, use the [Query](/operation/query) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var person = connection.Query(e => e.Id == 1); + /* Process the result here */ +} +``` + +To query all rows, use the [QueryAll](/operation/queryall) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var people = connection.QueryAll(); + /* Process the results here */ +} +``` + +{: .note } +> Vertica has no table-hint syntax (`AreTableHintsSupported` is `false`) — passing a `hints` argument to any operation throws a `NotSupportedException`. + +## Merging a Record + +To merge a row, use the [Merge](/operation/merge) method. + +```csharp +var person = new Person +{ + Id = 1, + Name = "John Doe", + Age = 57, + CreatedDateUtc = DateTime.UtcNow +}; +using (var connection = new VerticaConnection(ConnectionString)) +{ + var id = connection.Merge(person); +} +``` + +By default, the primary or identity column is used as a qualifier. Custom qualifiers can also be specified. + +```csharp +var person = new Person +{ + Name = "John Doe", + Age = 57, + CreatedDateUtc = DateTime.UtcNow +}; +using (var connection = new VerticaConnection(ConnectionString)) +{ + var id = connection.Merge(person, qualifiers: (p => new { p.Name })); +} +``` + +To merge multiple rows, use the [MergeAll](/operation/mergeall) method. + +```csharp +var people = GetPeople(100); +people + .AsList() + .ForEach(p => p.Name = $"{p.Name} (Merged)"); +using (var connection = new VerticaConnection(ConnectionString)) +{ + var affectedRecords = connection.MergeAll(people); +} +``` + +{: .important } +> Vertica flatly refuses to run a `MERGE` statement against a table with an `IDENTITY`/`AUTO_INCREMENT` column at all ("Sequence or IDENTITY/AUTO_INCREMENT column in merge query is not supported"), and has no procedural fallback equivalent to Firebird's `EXECUTE BLOCK`. [Merge](/operation/merge)/[MergeAll](/operation/mergeall) are instead always compiled as an `UPDATE ...` followed by an `INSERT ... WHERE NOT EXISTS (...)`, joined by `;` into a single command text — verify this against a live instance before relying on it in production, since `VerticaCommand` is documented elsewhere (see [Operations (Vertica)](/operation/vertica)) to refuse a compound statement that carries parameters. + +## Deleting a Record + +To delete a row, use the [Delete](/operation/delete) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var deletedRows = connection.Delete(1); +} +``` + +Other columns can also be used as qualifiers. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var deletedRows = connection.Delete(p => p.Name == "John Doe"); +} +``` + +To delete all rows, use the [DeleteAll](/operation/deleteall) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var deletedRows = connection.DeleteAll(); +} +``` + +{: .note } +> Both the [Delete](/operation/delete) and [DeleteAll](/operation/deleteall) methods return the number of rows affected during the execution. + +## Updating a Record + +To update a row, use the [Update](/operation/update) method. + +```csharp +var person = new Person +{ + Id = 1, + Name = "James Doe", + Age = 55, + CreatedDateUtc = DateTime.UtcNow +}; +using (var connection = new VerticaConnection(ConnectionString)) +{ + var updatedRows = connection.Update(person); +} +``` + +To update multiple rows, use the [UpdateAll](/operation/updateall) method. + +```csharp +var people = GetPeople(100); +people + .AsList() + .ForEach(p => p.Name = $"{p.Name} (Updated)"); +using (var connection = new VerticaConnection(ConnectionString)) +{ + var updatedRows = connection.UpdateAll(people); +} +``` + +{: .note } +> Both the [Update](/operation/update) and [UpdateAll](/operation/updateall) methods return the number of rows affected during the execution. + +## Executing a Query + +To execute a non-query statement, use the [ExecuteNonQuery](/operation/executenonquery) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var sql = "DELETE FROM \"Person\" WHERE \"Id\" = @Id"; + var affectedRecords = connection.ExecuteNonQuery(sql, new { Id = 1 }); +} +``` + +To execute a query and return mapped objects, use the [ExecuteQuery](/operation/executequery) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var sql = "SELECT * FROM \"Person\" ORDER BY \"Id\" ASC"; + var people = connection.ExecuteQuery(sql); + /* Process the results here */ +} +``` + +To execute a query and return a scalar value, use the [ExecuteScalar](/operation/executescalar) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var sql = "SELECT MAX(\"Id\") FROM \"Person\""; + var maxId = connection.ExecuteScalar(sql); +} +``` + +To execute a query and return a [DbDataReader](https://learn.microsoft.com/en-us/dotnet/api/system.data.common.dbdatareader?view=net-6.0), use the [ExecuteReader](/operation/executereader) method. + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var sql = "SELECT * FROM \"Person\""; + using (var reader = connection.ExecuteReader(sql)) + { + /* Process the data reader here */ + } +} +``` + +## Typed Result Execution + +Single-column result sets can be mapped to any .NET CLR type via [ExecuteQuery](/operation/executequery). + +```csharp +using (var connection = new VerticaConnection(ConnectionString)) +{ + var sql = "SELECT \"Name\" FROM \"Person\""; + var names = connection.ExecuteQuery(sql); +} +``` + +{: .note } +> The result of this operation is an [IEnumerable](https://learn.microsoft.com/en-us/dotnet/api/system.collections.generic.ienumerable-1?view=net-7.0) object. diff --git a/pages/home.md b/pages/home.md index 55e2e30..a383d56 100644 --- a/pages/home.md +++ b/pages/home.md @@ -34,6 +34,7 @@ Choose a database to get started quickly: - [SQL Server](/tutorial/get-started-sqlserver) - [SQLite](/tutorial/get-started-sqlite) - [Telemetry](/tutorial/get-started-telemetry) +- [Vertica](/tutorial/get-started-vertica) For a full topic index, visit the [docs](/docs) page. diff --git a/pages/operations/vertica/bulkdelete.md b/pages/operations/vertica/bulkdelete.md new file mode 100644 index 0000000..d5ce10c --- /dev/null +++ b/pages/operations/vertica/bulkdelete.md @@ -0,0 +1,156 @@ +--- +layout: default +sidebar: operations +title: "BulkDelete" +permalink: /operation/vertica/bulkdelete +tags: [repodb, tutorial, bulkdelete, orm, hybrid-orm, vertica] +parent: "Vertica" +grand_parent: OPERATIONS +--- + +# BulkDelete + +--- + +This method deletes rows from the database in bulk, matched by the defined qualifiers. It is supported for [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). Vertica also has a dedicated [BulkDeleteByKey](/operation/vertica/bulkdeletebykey) operation for deleting by primary key. + +{: .note } +> This page documents the Vertica-specific arguments and examples. For the SQL Server implementation, see [BulkDelete (SQL Server)](/operation/sqlserver/bulkdelete). + +## Call Flow Diagram + +The diagram below shows the flow when calling this operation. + +```mermaid +flowchart TD + Client["Client
(RepoDB)"] -->|BulkDelete| Source["Entities /
DataTable /
DbDataReader"] + Source --> Pseudo["Create Pseudo Table
(Auto/Memory/Physical) +
Index on qualifiers"] + Pseudo --> Stream["VerticaCopyStream
(COPY ... FROM STDIN)"] + Stream -->|Write| PseudoTable[("Pseudo Table
(qualifier columns only)")] + PseudoTable -->|"DELETE FROM Target
WHERE EXISTS (SELECT 1 FROM
Pseudo S WHERE qualifiers match)"| Table[("Target Table")] + PseudoTable -->|Drop| Cleanup(["Pseudo Table Dropped"]) +``` + +## Use Case + +Use this method to delete rows at high speed. It leverages `VerticaCopyStream`, `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +For deleting 1,000 or more rows, prefer this method over [DeleteAll](/operation/deleteall) — Vertica's `IsMultiStatementExecutable` setting is `false`, so [DeleteAll](/operation/deleteall) issues one round trip per row when deleting by a list of keys. + +A pseudo (staging) table, containing only the qualifier columns and indexed on them, is created for every call. The library streams into it via a `COPY` load internally, then cascades the deletions to the target table via a correlated `EXISTS` subquery — see [Operations (Vertica)](/operation/vertica) for the underlying mechanics. + +## Special Arguments + +The `qualifiers`, `bulkCopyTimeout`, `batchSize` and `pseudoTableType` arguments are available for this operation. + +`qualifiers` defines the fields used to match existing rows. Defaults to the primary or identity column if not specified. + +`bulkCopyTimeout` overrides the command timeout, in seconds. + +`batchSize` overrides the number of rows sent to the server per batch. When not set, all items are sent at once. + +`pseudoTableType` (via [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype)) controls the kind of staging table used internally. + +## Caveats + +This operation creates a pseudo (staging) table for every call — a per-call, uniquely-named `TABLE` or `GLOBAL TEMPORARY TABLE`, per `pseudoTableType`. The database user must have permission to create tables, or a `VerticaException` will be thrown. + +## Usability + +The following example retrieves all inactive people, then bulk-deletes them from the `Person` table. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = connection.Query(e => e.IsActive == false); + var deletedRows = connection.BulkDelete(people); +} +``` + +To specify a batch size: + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var deletedRows = connection.BulkDelete(people, batchSize: 100); +} +``` + +{: .note } +> When `batchSize` is not set, all rows are sent to the server in a single batch. + +#### DataTable + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var table = ConvertToDataTable(people); + var deletedRows = connection.BulkDelete("\"Person\"", table); +} +``` + +#### Dictionary/ExpandoObject + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + var result = sourceConnection.QueryAll("\"Person\""); + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var deletedRows = destinationConnection.BulkDelete("\"Person\"", result); + } +} +``` + +#### DataReader + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + using (var reader = sourceConnection.ExecuteReader("SELECT * FROM \"Person\"")) + { + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var rows = destinationConnection.BulkDelete("\"Person\"", reader); + } + } +} +``` + +## Targeting a Table + +To target a specific table, pass the literal table name. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var deletedRows = connection.BulkDelete("\"Person\"", people); +} +``` + +## Field Qualifiers + +By default, the primary or identity column is used as the qualifier. To override, pass a list of [Field](/class/field) objects in the `qualifiers` argument. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var deletedRows = connection.BulkDelete(people, + qualifiers: e => new { e.Name }); +} +``` + +{: .important } +> Use indexed columns from the target table as qualifiers to maximize performance. + +## Async Method + +An equivalent [BulkDeleteAsync](/operation/vertica/bulkdelete) method is also available. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = connection.Query(e => e.IsActive == false); + var deletedRows = await connection.BulkDeleteAsync(people); +} +``` diff --git a/pages/operations/vertica/bulkdeletebykey.md b/pages/operations/vertica/bulkdeletebykey.md new file mode 100644 index 0000000..808cb2a --- /dev/null +++ b/pages/operations/vertica/bulkdeletebykey.md @@ -0,0 +1,85 @@ +--- +layout: default +sidebar: operations +title: "BulkDeleteByKey" +permalink: /operation/vertica/bulkdeletebykey +tags: [repodb, tutorial, bulkdeletebykey, orm, hybrid-orm, vertica] +parent: "Vertica" +grand_parent: OPERATIONS +--- + +# BulkDeleteByKey + +--- + +This method deletes rows from the database using a list of primary keys in bulk. It is supported for [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). + +## Call Flow Diagram + +The diagram below shows the flow when calling this operation. + +```mermaid +flowchart TD + Client["Client
(RepoDB)"] -->|BulkDeleteByKey| Keys["Primary Keys
IEnumerable<TPrimaryKey>"] + Keys --> Pseudo["Create Pseudo Table
(Auto/Memory/Physical) +
Index on key column"] + Pseudo --> Stream["VerticaCopyStream
(COPY ... FROM STDIN)"] + Stream -->|Write| PseudoTable[("Pseudo Table
(key column only)")] + PseudoTable -->|"DELETE FROM Target
WHERE EXISTS (SELECT 1 FROM
Pseudo S WHERE key matches)"| Table[("Target Table")] + PseudoTable -->|Drop| Cleanup(["Pseudo Table Dropped"]) +``` + +## Use Case + +Use this method to delete rows by primary key at high speed. It leverages `VerticaCopyStream`, `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +## Special Arguments + +The `bulkCopyTimeout`, `batchSize` and `pseudoTableType` arguments are available for this operation. + +`bulkCopyTimeout` overrides the command timeout, in seconds. + +`batchSize` overrides the number of rows sent to the server per batch. When not set, all items are sent at once. + +`pseudoTableType` (via [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype)) controls the kind of staging table used internally. + +## Caveats + +This operation creates a pseudo (staging) table for every call — a per-call, uniquely-named `TABLE` or `GLOBAL TEMPORARY TABLE`, per `pseudoTableType`. The database user must have permission to create tables, or a `VerticaException` will be thrown. + +## Usability + +Pass the target table name and the list of primary keys to the operation. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var primaryKeys = connection.Query(p => p.IsActive == false).Select(p => p.Id); + var deletedRows = connection.BulkDeleteByKey(primaryKeys); +} +``` + +{: .note } +> It returns the number of rows deleted from the underlying table. + +To specify a batch size: + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var primaryKeys = connection.Query(p => p.IsActive == false).Select(p => p.Id); + var deletedRows = connection.BulkDeleteByKey(primaryKeys, + batchSize: 100); +} +``` + +## Async Method + +An equivalent [BulkDeleteByKeyAsync](/operation/vertica/bulkdeletebykey) method is also available. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var primaryKeys = connection.Query(p => p.IsActive == false).Select(p => p.Id); + var deletedRows = await connection.BulkDeleteByKeyAsync(primaryKeys); +} +``` diff --git a/pages/operations/vertica/bulkinsert.md b/pages/operations/vertica/bulkinsert.md new file mode 100644 index 0000000..0af6cfe --- /dev/null +++ b/pages/operations/vertica/bulkinsert.md @@ -0,0 +1,221 @@ +--- +layout: default +sidebar: operations +title: "BulkInsert" +permalink: /operation/vertica/bulkinsert +tags: [repodb, tutorial, bulkinsert, orm, hybrid-orm, vertica] +parent: "Vertica" +grand_parent: OPERATIONS +--- + +# BulkInsert + +--- + +This method inserts all rows from the client application into the database in bulk. It is supported for [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). + +{: .note } +> This page documents the Vertica-specific arguments and examples. For the SQL Server implementation, see [BulkInsert (SQL Server)](/operation/sqlserver/bulkinsert). + +## Call Flow Diagram + +The diagram below shows the flow when calling this operation. + +```mermaid +flowchart TD + Client["Client
(RepoDB)"] -->|BulkInsert| Source["Entities /
DataTable /
DbDataReader"] + Source --> Decision{"identityBehavior ==
ReturnIdentity?"} + Decision -->|NO| Direct["VerticaCopyStream
(COPY ... FROM STDIN)"] + Direct -->|Write| Table[("Target Table")] + Decision -->|YES| Pseudo["Create Pseudo Table
(Auto/Memory/Physical)"] + Pseudo --> Staged["VerticaCopyStream
(COPY ... FROM STDIN)"] + Staged -->|Write| PseudoTable[("Pseudo Table")] + PseudoTable --> Insert["INSERT INTO Target (...)
SELECT ... FROM Pseudo
ORDER BY row-order"] + Insert --> Table + Insert -->|"SELECT LAST_INSERT_ID();
back-compute per-row values"| Client + PseudoTable -->|Drop| Cleanup(["Pseudo Table Dropped"]) +``` + +## Use Case + +Use this method to insert rows at high speed. It leverages `VerticaCopyStream`, `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +For inserting 1,000 or more rows, prefer this method over [InsertAll](/operation/insertall). + +Rows are written straight to the target table. A pseudo (staging) table is only used when `identityBehavior` is set to `ReturnIdentity` (see below) — see [Operations (Vertica)](/operation/vertica) for the underlying mechanics. + +## Special Arguments + +The `mappings`, `bulkCopyTimeout`, `batchSize`, `identityBehavior` and `pseudoTableType` arguments are available for this operation. + +`mappings` (via `VerticaBulkInsertMapItem`) defines explicit column mappings between the source properties and the destination columns. When omitted, columns are auto-mapped by name (case-insensitive). + +`bulkCopyTimeout` overrides the command timeout, in seconds. + +`batchSize` overrides the number of rows sent to the server per batch. When not set, all items are sent at once. + +`identityBehavior` (via [VerticaBulkImportIdentityBehavior](/enumeration/vertica/verticabulkimportidentitybehavior)) controls whether newly generated identity values are set back on the data entities. Disabled (`KeepIdentity`) by default. Enabling this (`ReturnIdentity`) routes the operation through a pseudo table instead. + +`pseudoTableType` (via [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype)) controls the kind of staging table used when `identityBehavior` is `ReturnIdentity`. + +{: .note } +> The `DbDataReader` overload has no `identityBehavior` argument — a forward-only, single-pass reader cannot be rewound to correlate generated identity values back onto a source row, so `ReturnIdentity` is not supported for that overload. + +## Identity Setting Alignment + +When `identityBehavior` is `ReturnIdentity`, the pseudo-table rows are inserted into the real table via a single `INSERT INTO ... SELECT ... FROM Pseudo ORDER BY __RepoDbBulkRowOrder__` statement, then `SELECT LAST_INSERT_ID()` is read once. Because Vertica assigns `IDENTITY`/`AUTO_INCREMENT` values contiguously in insertion order (and the `INSERT`'s own `SELECT` is itself ordered by the pseudo table's row-order column), every row's actual identity is reconstructed by subtracting a descending offset from that single last-identity value, rather than requiring one round trip per row. + +{: .important } +> This technique — and `GetScopeIdentity`'s underlying `SELECT LAST_INSERT_ID()` query — has not been verified against a live Vertica instance. Verify it end-to-end, especially under concurrent writers to the same table, before relying on it in production. + +## Usability + +The following example defines a method that produces a list of `Person` objects, then bulk-inserts 10,000 rows into the `Person` table. + +```csharp +private IEnumerable GetPeople(int count = 1000) +{ + for (var i = 0; i < count; i++) + { + yield return new Person + { + Name = $"Person-{i}", + Age = 30, + CreatedDateUtc = DateTime.UtcNow + }; + } +} +``` + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var insertedRows = connection.BulkInsert(people); +} +``` + +To specify a batch size: + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var insertedRows = connection.BulkInsert(people, batchSize: 100); +} +``` + +{: .note } +> When `batchSize` is not set, all rows are sent to the server in a single batch. + +To return the newly generated identity values: + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var insertedRows = connection.BulkInsert(people, + identityBehavior: VerticaBulkImportIdentityBehavior.ReturnIdentity); +} +``` + +#### DataTable + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var table = ConvertToDataTable(people); + var insertedRows = connection.BulkInsert("\"Person\"", table); +} +``` + +#### Dictionary/ExpandoObject + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + var result = sourceConnection.QueryAll("\"Person\""); + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var insertedRows = destinationConnection.BulkInsert("\"Person\"", result); + } +} +``` + +#### DataReader + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + using (var reader = sourceConnection.ExecuteReader("SELECT * FROM \"Person\"")) + { + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var rows = destinationConnection.BulkInsert("\"Person\"", reader); + } + } +} +``` + +To bulk-insert via [DataEntityDataReader](/class/dataentitydatareader): + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + using (var reader = new DataEntityDataReader(people)) + { + var insertedRows = connection.BulkInsert("\"Person\"", reader); + } +} +``` + +## Column Mappings + +Add column mappings using the `VerticaBulkInsertMapItem` class. + +```csharp +var mappings = new List(); + +// Add the mappings +mappings.Add(new VerticaBulkInsertMapItem("SourceId", "DestinationId")); +mappings.Add(new VerticaBulkInsertMapItem("SourceName", "DestinationName")); +mappings.Add(new VerticaBulkInsertMapItem("SourceAge", "DestinationAge")); +mappings.Add(new VerticaBulkInsertMapItem("SourceCreatedDateUtc", "DestinationCreatedDateUtc")); + +// Execute +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var insertedRows = connection.BulkInsert(people, + mappings: mappings); +} +``` + +## Targeting a Table + +To target a specific table, pass the literal table name. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var insertedRows = connection.BulkInsert("\"Person\"", people); +} +``` + +## Async Method + +An equivalent [BulkInsertAsync](/operation/vertica/bulkinsert) method is also available. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var insertedRows = await connection.BulkInsertAsync(people); +} +``` + +{: .note } +> `VerticaCopyStream` exposes no async API of its own, so the async overload offloads the synchronous `Start`/`AddStream`/`Execute`/`Finish` sequence to a background thread instead. diff --git a/pages/operations/vertica/bulkmerge.md b/pages/operations/vertica/bulkmerge.md new file mode 100644 index 0000000..e462be5 --- /dev/null +++ b/pages/operations/vertica/bulkmerge.md @@ -0,0 +1,214 @@ +--- +layout: default +sidebar: operations +title: "BulkMerge" +permalink: /operation/vertica/bulkmerge +tags: [repodb, tutorial, bulkmerge, orm, hybrid-orm, vertica] +parent: "Vertica" +grand_parent: OPERATIONS +--- + +# BulkMerge + +--- + +This method merges all rows from the client application into the database in bulk — inserting new rows and updating existing ones based on the defined qualifiers. It is supported for [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). + +{: .note } +> This page documents the Vertica-specific arguments and examples. For the SQL Server implementation, see [BulkMerge (SQL Server)](/operation/sqlserver/bulkmerge). + +## Call Flow Diagram + +The diagram below shows the flow when calling this operation. + +```mermaid +flowchart TD + Client["Client
(RepoDB)"] -->|BulkMerge| Source["Entities /
DataTable /
DbDataReader"] + Source --> Pseudo["Create Pseudo Table
(Auto/Memory/Physical) +
Index on qualifiers"] + Pseudo --> Stream["VerticaCopyStream
(COPY ... FROM STDIN)"] + Stream -->|Write| PseudoTable[("Pseudo Table")] + PseudoTable --> Update["UPDATE Target SET ...
FROM Pseudo WHERE qualifiers match
(skipped if nothing to update)"] + Update --> Insert["INSERT INTO Target (...)
SELECT ... FROM Pseudo
WHERE NOT EXISTS (...)
ORDER BY row-order"] + Insert --> Table[("Target Table")] + Update --> Table + Insert -->|"identity is qualifier:
back-compute new rows' identities.
otherwise: re-SELECT by qualifier join"| Client + PseudoTable -->|Drop| Cleanup(["Pseudo Table Dropped"]) +``` + +## Use Case + +Use this method to merge rows at high speed. It leverages `VerticaCopyStream`, `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +For merging 1,000 or more rows, prefer this method over [MergeAll](/operation/mergeall) — Vertica's `IsMultiStatementExecutable` setting is `false`, so [MergeAll](/operation/mergeall) issues one round trip per row. + +A pseudo (staging) table, indexed on the qualifier columns, is created for every call. The library streams into it via a `COPY` load internally, then cascades the changes to the target table — see [Operations (Vertica)](/operation/vertica) for the underlying mechanics. + +## Special Arguments + +The `qualifiers`, `mappings`, `bulkCopyTimeout`, `batchSize`, `identityBehavior` and `pseudoTableType` arguments are available for this operation. + +`qualifiers` defines the fields used to match existing rows. Defaults to the primary or identity column if not specified. + +`mappings` (via `VerticaBulkInsertMapItem`) defines explicit column mappings between the source properties and the destination columns. When omitted, columns are auto-mapped by name (case-insensitive). + +`bulkCopyTimeout` overrides the command timeout, in seconds. + +`batchSize` overrides the number of rows sent to the server per batch. When not set, all items are sent at once. + +`identityBehavior` (via [VerticaBulkImportIdentityBehavior](/enumeration/vertica/verticabulkimportidentitybehavior)) controls whether newly generated identity values are set back on the data entities. Disabled (`KeepIdentity`) by default. + +`pseudoTableType` (via [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype)) controls the kind of staging table used internally. + +{: .note } +> The `DbDataReader` overload has no `identityBehavior` argument, for the same reason as [BulkInsert](/operation/vertica/bulkinsert)'s reader overload. + +## Operation SQL Statements + +Vertica flatly refuses to run a `MERGE` statement at all against a table that has an `IDENTITY`/`AUTO_INCREMENT` column ("Sequence or IDENTITY/AUTO_INCREMENT column in merge query is not supported"), and has no procedural fallback equivalent to Firebird's `EXECUTE BLOCK`. A bulk merge is instead always two separate statements: + +1. `UPDATE Target SET ... FROM Pseudo S WHERE qualifiers match` — skipped entirely if there are no non-qualifier, non-identity fields to update. +2. `INSERT INTO Target (...) SELECT ... FROM Pseudo S WHERE NOT EXISTS (SELECT 1 FROM Target WHERE qualifiers match) ORDER BY S.__RepoDbBulkRowOrder__` — the identity column, if any, is always excluded, and the explicit `ORDER BY` keeps the newly-inserted rows in source order (without it, Vertica is free to insert unmatched rows in whatever order its projections yield them). + +When `identityBehavior` is `ReturnIdentity`: + +- If the identity column is itself a qualifier, a row's original identity value doubles as caller intent: a real, already-known value means "update this existing row," and an unset `0`/`null` sentinel means "insert a new row, generate its identity." New rows' identities are back-computed from a single `SELECT LAST_INSERT_ID()` the same way [BulkInsert](/operation/vertica/bulkinsert) does, since Vertica assigns them contiguously in the `INSERT`'s own row order. +- Otherwise, every row's identity (whether pre-existing or newly generated) is read back afterward via a join between the pseudo table and the target table on the qualifier columns, ordered by the row-order column. + +{: .note } +> Unlike the plain (non-bulk) [Merge](/operation/merge) operation, these two statements are executed as two separate round trips — not joined into one compound command text — so `IsMultiStatementExecutable` being `false` does not affect this path. + +## Usability + +Given a list of `Person` models containing both existing and new rows, the following example bulk-merges them into the `Person` table. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var mergedRows = connection.BulkMerge(people); +} +``` + +To specify a batch size: + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var mergedRows = connection.BulkMerge(people, batchSize: 100); +} +``` + +{: .note } +> When `batchSize` is not set, all rows are sent to the server in a single batch. + +#### DataTable + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var table = ConvertToDataTable(people); + var mergedRows = connection.BulkMerge("\"Person\"", table); +} +``` + +#### Dictionary/ExpandoObject + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + var result = sourceConnection.QueryAll("\"Person\""); + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var mergedRows = destinationConnection.BulkMerge("\"Person\"", result, + qualifiers: Field.From("Name")); + } +} +``` + +#### DataReader + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + using (var reader = sourceConnection.ExecuteReader("SELECT * FROM \"Person\" WHERE \"Age\" > 18")) + { + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var rows = destinationConnection.BulkMerge("\"Person\"", reader); + } + } +} +``` + +To bulk-merge via [DataEntityDataReader](/class/dataentitydatareader): + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + using (var reader = new DataEntityDataReader(people)) + { + var mergedRows = connection.BulkMerge("\"Person\"", reader); + } +} +``` + +## Field Qualifiers + +By default, the primary or identity column is used as the qualifier. To override, pass a list of [Field](/class/field) objects in the `qualifiers` argument. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var mergedRows = connection.BulkMerge(people, + qualifiers: e => new { e.Name }); +} +``` + +{: .important } +> Use indexed columns from the target table as qualifiers to maximize performance. + +## Column Mappings + +Add column mappings using the `VerticaBulkInsertMapItem` class. + +```csharp +var mappings = new List(); + +// Add the mappings +mappings.Add(new VerticaBulkInsertMapItem("SourceId", "DestinationId")); +mappings.Add(new VerticaBulkInsertMapItem("SourceName", "DestinationName")); +mappings.Add(new VerticaBulkInsertMapItem("SourceAge", "DestinationAge")); +mappings.Add(new VerticaBulkInsertMapItem("SourceCreatedDateUtc", "DestinationCreatedDateUtc")); + +// Execute +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var mergedRows = connection.BulkMerge(people, + mappings: mappings); +} +``` + +## Targeting a Table + +To target a specific table, pass the literal table name. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var mergedRows = connection.BulkMerge("\"Person\"", people); +} +``` + +## Async Method + +An equivalent [BulkMergeAsync](/operation/vertica/bulkmerge) method is also available. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var mergedRows = await connection.BulkMergeAsync(people); +} +``` diff --git a/pages/operations/vertica/bulkupdate.md b/pages/operations/vertica/bulkupdate.md new file mode 100644 index 0000000..fe27ef0 --- /dev/null +++ b/pages/operations/vertica/bulkupdate.md @@ -0,0 +1,197 @@ +--- +layout: default +sidebar: operations +title: "BulkUpdate" +permalink: /operation/vertica/bulkupdate +tags: [repodb, tutorial, bulkupdate, orm, hybrid-orm, vertica] +parent: "Vertica" +grand_parent: OPERATIONS +--- + +# BulkUpdate + +--- + +This method updates existing rows in the database in bulk, matched by the defined qualifiers. It is supported for [Vertica](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations). + +{: .note } +> This page documents the Vertica-specific arguments and examples. For the SQL Server implementation, see [BulkUpdate (SQL Server)](/operation/sqlserver/bulkupdate). + +## Call Flow Diagram + +The diagram below shows the flow when calling this operation. + +```mermaid +flowchart TD + Client["Client
(RepoDB)"] -->|BulkUpdate| Source["Entities /
DataTable /
DbDataReader"] + Source --> Pseudo["Create Pseudo Table
(Auto/Memory/Physical) +
Index on qualifiers"] + Pseudo --> Stream["VerticaCopyStream
(COPY ... FROM STDIN)"] + Stream -->|Write| PseudoTable[("Pseudo Table")] + PseudoTable -->|"UPDATE Target SET ...
FROM Pseudo WHERE qualifiers match
(no INSERT step)"| Table[("Target Table")] + PseudoTable -->|Drop| Cleanup(["Pseudo Table Dropped"]) +``` + +## Use Case + +Use this method to update rows at high speed. It leverages `VerticaCopyStream`, `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +For updating 1,000 or more rows, prefer this method over [UpdateAll](/operation/updateall) — Vertica's `IsMultiStatementExecutable` setting is `false`, so [UpdateAll](/operation/updateall) issues one round trip per row. + +A pseudo (staging) table, indexed on the qualifier columns, is created for every call. The library streams into it via a `COPY` load internally, then cascades the changes to the target table via an `UPDATE ... FROM` statement with no `INSERT` step — staged rows with no matching target row are left as-is, not inserted. This reuses the same SQL-generation path as the update half of [BulkMerge](/operation/vertica/bulkmerge#operation-sql-statements), so Vertica's identity-column restrictions apply here too. See [Operations (Vertica)](/operation/vertica) for the underlying mechanics. + +## Special Arguments + +The `qualifiers`, `mappings`, `bulkCopyTimeout`, `batchSize` and `pseudoTableType` arguments are available for this operation. + +`qualifiers` defines the fields used to match existing rows. Defaults to the primary or identity column if not specified. + +`mappings` (via `VerticaBulkInsertMapItem`) defines explicit column mappings between the source properties and the destination columns. When omitted, columns are auto-mapped by name (case-insensitive). + +`bulkCopyTimeout` overrides the command timeout, in seconds. + +`batchSize` overrides the number of rows sent to the server per batch. When not set, all items are sent at once. + +`pseudoTableType` (via [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype)) controls the kind of staging table used internally. + +{: .note } +> If every staged field is also a qualifier (nothing left to actually update), the generated `UPDATE` is skipped and the operation returns `0`. + +## Caveats + +This operation creates a pseudo (staging) table for every call — a per-call, uniquely-named `TABLE` or `GLOBAL TEMPORARY TABLE`, per `pseudoTableType`. The database user must have permission to create tables, or a `VerticaException` will be thrown. + +## Usability + +Given a list of `Person` models, the following example bulk-updates rows in the `Person` table. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var updatedRows = connection.BulkUpdate(people); +} +``` + +To specify a batch size: + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var updatedRows = connection.BulkUpdate(people, batchSize: 100); +} +``` + +{: .note } +> When `batchSize` is not set, all rows are sent to the server in a single batch. + +#### DataTable + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var table = ConvertToDataTable(people); + var updatedRows = connection.BulkUpdate("\"Person\"", table); +} +``` + +#### Dictionary/ExpandoObject + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + var result = sourceConnection.QueryAll("\"Person\""); + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var updatedRows = destinationConnection.BulkUpdate("\"Person\"", result, + qualifiers: Field.From("Name")); + } +} +``` + +#### DataReader + +```csharp +using (var sourceConnection = new VerticaConnection(sourceConnectionString)) +{ + using (var reader = sourceConnection.ExecuteReader("SELECT * FROM \"Person\" WHERE \"Age\" > 18")) + { + using (var destinationConnection = new VerticaConnection(destinationConnectionString)) + { + var rows = destinationConnection.BulkUpdate("\"Person\"", reader); + } + } +} +``` + +To bulk-update via [DataEntityDataReader](/class/dataentitydatareader): + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + using (var reader = new DataEntityDataReader(people)) + { + var updatedRows = connection.BulkUpdate("\"Person\"", reader); + } +} +``` + +## Field Qualifiers + +By default, the primary or identity column is used as the qualifier. To override, pass a list of [Field](/class/field) objects in the `qualifiers` argument. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var updatedRows = connection.BulkUpdate(people, + qualifiers: e => new { e.Name }); +} +``` + +{: .important } +> Use indexed columns from the target table as qualifiers to maximize performance. + +## Column Mappings + +Add column mappings using the `VerticaBulkInsertMapItem` class. + +```csharp +var mappings = new List(); + +// Add the mappings +mappings.Add(new VerticaBulkInsertMapItem("SourceId", "DestinationId")); +mappings.Add(new VerticaBulkInsertMapItem("SourceName", "DestinationName")); +mappings.Add(new VerticaBulkInsertMapItem("SourceAge", "DestinationAge")); +mappings.Add(new VerticaBulkInsertMapItem("SourceCreatedDateUtc", "DestinationCreatedDateUtc")); + +// Execute +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var updatedRows = connection.BulkUpdate(people, + mappings: mappings); +} +``` + +## Targeting a Table + +To target a specific table, pass the literal table name. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var people = GetPeople(10000); + var updatedRows = connection.BulkUpdate("\"Person\"", people); +} +``` + +## Async Method + +An equivalent [BulkUpdateAsync](/operation/vertica/bulkupdate) method is also available. + +```csharp +using (var connection = new VerticaConnection(connectionString)) +{ + var updatedRows = await connection.BulkUpdateAsync(people); +} +``` diff --git a/pages/operations/vertica/vertica.md b/pages/operations/vertica/vertica.md new file mode 100644 index 0000000..bbf8474 --- /dev/null +++ b/pages/operations/vertica/vertica.md @@ -0,0 +1,122 @@ +--- +layout: default +title: "Vertica" +has_children: true +permalink: /operation/vertica +parent: OPERATIONS +--- + +# Operations (Vertica) + +--- + +RepoDB's standard operations ([Query](/operation/query), [Insert](/operation/insert), [Merge](/operation/merge), [Update](/operation/update), [Delete](/operation/delete), etc.) all work against `VerticaConnection` once [UseVertica()](/class/vertica/verticaconfiguration) has been called. `BulkInsert`, `BulkMerge`, `BulkUpdate`, `BulkDelete` and `BulkDeleteByKey` are provided by the separate [RepoDb.Vertica.BulkOperations](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) package, built on `VerticaCopyStream` — `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +For [BulkDelete](/operation/vertica/bulkdelete), [BulkDeleteByKey](/operation/vertica/bulkdeletebykey), [BulkMerge](/operation/vertica/bulkmerge) and [BulkUpdate](/operation/vertica/bulkupdate), a pseudo (staging) table is created — and dropped — for every call, indexed on the qualifier columns. The library streams into it via a `COPY` load internally, then cascades the changes to the original table using the correct SQL statement. + +For [BulkInsert](/operation/vertica/bulkinsert), rows are streamed straight to the target table — unless [VerticaBulkImportIdentityBehavior.ReturnIdentity](/enumeration/vertica/verticabulkimportidentitybehavior) is requested, in which case a pseudo table is used first so the generated identity values can be read back. + +{: .note } +> Every pseudo table gets a per-call unique name, so unlike some other providers' bulk-operations packages, concurrent callers writing against the same target table never race on a shared staging-table name. + +The other bulk operations can be optimized further by targeting the underlying table indexes (via qualifiers). Pass a list of [Field](/class/field) objects when calling the operations. + +## Pseudo Table Type + +The [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype) enum lets you choose between a Vertica `GLOBAL TEMPORARY TABLE` (`Memory`), an ordinary heap table (`Physical`), or let the library decide based on row count (`Auto`, the default — `Physical` at 5,000 rows or more, otherwise `Memory`). + +## Supported Objects + +Below are the following objects supported by the bulk operations. + +- System.DataTable +- System.Data.Common.DbDataReader +- IEnumerable<T> +- ExpandoObject +- IDictionary<string, object> + +## Operation SQL Statements + +Once all the data is in the staging (pseudo) table, the correct SQL statement is used to cascade the changes towards the original table. + +{: .note } +> [BulkInsert](/operation/vertica/bulkinsert) writes directly into the target table and skips the staging table entirely — unless `identityBehavior` is set to `ReturnIdentity`, in which case a staging table is used first (see above). + +#### For BulkDelete / BulkDeleteByKey + +```csharp +> DELETE FROM "OriginalTable" +> WHERE EXISTS ( +> SELECT 1 FROM "PseudoTempTable" S +> WHERE "OriginalTable".QualifierField1 = S.QualifierField1 AND "OriginalTable".QualifierField2 = S.QualifierField2 +> ); +``` + +#### For BulkMerge + +Vertica flatly refuses to run a `MERGE` statement at all against a table that has an `IDENTITY`/`AUTO_INCREMENT` column, so a bulk merge is always expressed as two separate statements against the pseudo table — never a native `MERGE`. + +```csharp +> UPDATE "OriginalTable" SET Field3 = S.Field3, Field4 = S.Field4 +> FROM "PseudoTempTable" S +> WHERE "OriginalTable".QualifierField1 = S.QualifierField1 AND "OriginalTable".QualifierField2 = S.QualifierField2; +> +> INSERT INTO "OriginalTable" (Field1, Field2, ...) +> SELECT S.Field1, S.Field2, ... FROM "PseudoTempTable" S +> WHERE NOT EXISTS ( +> SELECT 1 FROM "OriginalTable" +> WHERE "OriginalTable".QualifierField1 = S.QualifierField1 AND "OriginalTable".QualifierField2 = S.QualifierField2 +> ) +> ORDER BY S.__RepoDbBulkRowOrder__; +``` + +{: .note } +> The `UPDATE` step is skipped entirely when there are no non-qualifier, non-identity fields left to update. Unlike the plain (non-bulk) [Merge](/operation/merge) operation, these two statements are executed as two separate round trips — not joined into one compound command text — so `IsMultiStatementExecutable` being `false` does not affect this path. + +#### For BulkUpdate + +```csharp +> UPDATE "OriginalTable" SET Field3 = S.Field3, Field4 = S.Field4 +> FROM "PseudoTempTable" S +> WHERE "OriginalTable".QualifierField1 = S.QualifierField1 AND "OriginalTable".QualifierField2 = S.QualifierField2; +``` + +{: .note } +> Unlike [BulkMerge](/operation/vertica/bulkmerge), there is no `INSERT` step — staged rows with no matching target row are left as-is, not inserted. + +## Special Arguments + +The arguments below are available on most operations. + +| Argument | Description | +|:---------|:------------| +| `qualifiers` | Defines the fields used to match existing rows. Defaults to the primary or identity column when not provided. | +| `identityBehavior` | Via [VerticaBulkImportIdentityBehavior](/enumeration/vertica/verticabulkimportidentitybehavior), controls whether the identity property is kept as-is, or whether newly generated identity values are returned back to the entities after [BulkInsert](/operation/vertica/bulkinsert) or [BulkMerge](/operation/vertica/bulkmerge). | +| `pseudoTableType` | Via [VerticaBulkImportPseudoTableType](/enumeration/vertica/verticabulkimportpseudotabletype), controls the kind of staging table created — see [Pseudo Table Type](#pseudo-table-type) above. | +| `batchSize` | Overrides the number of rows sent to the server per batch. When not set, all items are sent at once. | + +## BatchSize + +All the provided bulk operations have a `batchSize` argument that lets you override the number of rows wired-up to the server per batch. By default it is `null`, meaning all items are sent together in one go. + +Use this argument if you wish to optimize the operation based on certain situations. + +- Network Latency +- Infrastructure +- No. of Columns +- Type of Data + +## No table hints + +`VerticaDbSetting.AreTableHintsSupported` is `false`. Passing a non-null `hints` argument to any operation throws a `NotSupportedException`. + +## No true multi-statement execution, except for InsertAll + +`VerticaDbSetting.IsMultiStatementExecutable` is `false` — `VerticaCommand` refuses to execute a compound `;`-separated statement once it carries a parameter. [MergeAll](/operation/mergeall)/[UpdateAll](/operation/updateall) issue one statement per row instead of a single batched command; passing an explicit `batchSize` greater than `1` to either throws a `NotSupportedException`. [InsertAll](/operation/insertall) is the exception — `VerticaDbSetting.IsInsertAllBatchable` is `true`, so it still batches multiple rows into one genuine multi-row `INSERT ... VALUES (...), (...), ...` statement. + +{: .important } +> The plain (non-bulk) [Merge](/operation/merge)/[MergeAll](/operation/mergeall) operations compile an `UPDATE ...; INSERT ...` compound statement carrying parameters in both halves — this appears to conflict with the restriction above, and has not been verified against a live Vertica instance. See [VerticaStatementBuilder](/class/vertica/verticastatementbuilder) for details. + +## Async Methods + +All the provided synchronous operations have an equivalent asynchronous (`Async`) counterpart. diff --git a/pages/releases/vertica.md b/pages/releases/vertica.md new file mode 100644 index 0000000..b6c6c0c --- /dev/null +++ b/pages/releases/vertica.md @@ -0,0 +1,50 @@ +--- +layout: default +sidebar: releases +title: Vertica +description: "This page contains the latest information of the releases of RepoDb.Vertica library." +permalink: /release/vertica +parent: RELEASES +--- + +# Releases for RepoDb.Vertica + +--- + +View the NuGet package [here](https://www.nuget.org/packages/RepoDb.Vertica) or download it directly [here](https://www.nuget.org/api/v2/package/RepoDb.Vertica). + +## RepoDb.Vertica (v0.0.1-alpha) - Preview + +Released: TBA + +New +{: .label .label-green } + +First (alpha) release of the Vertica provider for RepoDB, built on top of [Vertica.Data](https://www.nuget.org/packages/Vertica.Data). Targets netstandard2.0, .NET 8, .NET 9, and .NET 10. + +> **Verification status:** this package has been implemented and reviewed, but not yet exercised against a live Vertica instance. Verify the Merge/MergeAll compound-statement behavior in particular (see Known limitations below) before relying on this package in production. + +### What's included + +- [VerticaBootstrap](/class/vertica/verticabootstrap) / [VerticaConfiguration](/class/vertica/verticaconfiguration)`.UseVertica()` — initializes RepoDB for use with `VerticaConnection` (`GlobalConfiguration.Setup().UseVertica()`), mapping the [VerticaDbSetting](/class/vertica/verticadbsetting), [VerticaDbHelper](/class/vertica/verticadbhelper), and [VerticaStatementBuilder](/class/vertica/verticastatementbuilder) to every `VerticaConnection`. Also forces `CultureInfo.CurrentCulture` to `CultureInfo.InvariantCulture` process-wide, working around `Vertica.Data` formatting date-like values using the ambient thread culture. +- [VerticaDbSetting](/class/vertica/verticadbsetting) — Vertica-specific behavior: `"` for opening/closing identifier quotes, `@` parameter prefix, no table hints, no multi-statement command text support (`IsMultiStatementExecutable = false`) except for [InsertAll](/operation/insertall) (`IsInsertAllBatchable = true`), a lower `MaxParameterCount` of `1500`, and both `RequiresDbTypeBeforeValue` and `SkipsUnreferencedParameters` set to `true` (see the [Core release notes](/release/core) for what those two control). +- [VerticaStatementBuilder](/class/vertica/verticastatementbuilder) — generates Vertica-flavored SQL for every operation: `LIMIT`/`LIMIT ... OFFSET` paging, a genuine multi-row `VALUES` list for [InsertAll](/operation/insertall), and an `UPDATE ...; INSERT ... WHERE NOT EXISTS (...)` pair (with a follow-up `SELECT LAST_INSERT_ID()` when a key is needed) for [Merge](/operation/merge)/[MergeAll](/operation/mergeall) — never a native `MERGE`, since Vertica rejects that statement outright against any table with an `IDENTITY`/`AUTO_INCREMENT` column. [Truncate](/operation/truncate) compiles to a plain `DELETE FROM t`, since Vertica has no `TRUNCATE TABLE` statement. +- [VerticaDbHelper](/class/vertica/verticadbhelper) — schema/type discovery (`GetFields`/`GetFieldsAsync`) sourced from `v_catalog.columns`/`v_catalog.primary_keys`. `GetScopeIdentity`/`GetScopeIdentityAsync` run `SELECT LAST_INSERT_ID()`. +- Resolvers — [VerticaConvertFieldResolver](/class/vertica/verticaconvertfieldresolver) (casts a field to its Vertica type when a conversion is needed, e.g. typed [ExecuteQuery](/operation/executequery) results), [VerticaDbTypeNameToClientTypeResolver](/class/vertica/verticadbtypenametoclienttyperesolver) (maps the type names returned by `v_catalog.columns` to .NET CLR types — widened to `long`/`double` since Vertica has no distinct integer/float storage widths), [DbTypeToVerticaStringNameResolver](/class/vertica/dbtypetoverticastringnameresolver) (maps `System.Data.DbType` to Vertica SQL type names, e.g. `DbType.Guid` → `UUID`), and [DbTypeNameToColumnNameResolver](/class/vertica/dbtypenametocolumnnameresolver) (maps a Vertica database type name to its base column type keyword; used internally by [RepoDb.Vertica.BulkOperations](/release/verticabulk) to generate pseudo-table column definitions). +- [TimeToDateTimePropertyHandler](/class/vertica/timetodatetimepropertyhandler) — re-bases the date component of a value read back from a `TIME` column onto `DateTime`'s default date, since Vertica's driver returns a `TIME` value combined with today's date rather than a fixed placeholder. +- Parameter attributes mirroring `VerticaParameter` members, settable per entity property: [VerticaType](/attribute/vertica/verticatype), [SourceColumn](/attribute/vertica/sourcecolumn), [SourceColumnNullMapping](/attribute/vertica/sourcecolumnnullmapping), and [SourceVersion](/attribute/vertica/sourceversion). + +### Known limitations (v1) + +- [Merge](/operation/merge)/[MergeAll](/operation/mergeall) compile a compound `UPDATE ...; INSERT ...` command text — both halves carrying parameters — joined by `;` into a single string submitted through `VerticaCommand.CommandText`. This has not been verified against a live Vertica instance, and appears to conflict with the very restriction `IsInsertAllBatchable`'s own remarks describe elsewhere: "a compound (\"stmt1; stmt2\") statement, which `VerticaCommand` refuses to execute once it carries a parameter." Verify this specifically before relying on [Merge](/operation/merge)/[MergeAll](/operation/mergeall) in production. +- `GetScopeIdentity`/`GetScopeIdentityAsync` run `SELECT LAST_INSERT_ID()` — unverified against a live instance. +- [MergeAll](/operation/mergeall)/[UpdateAll](/operation/updateall) issue a separate round trip per row rather than a single batched statement, since `IsMultiStatementExecutable` is `false` and neither has an `IsInsertAllBatchable`-style override. Passing an explicit `batchSize` greater than `1` to either throws a `NotSupportedException`. [InsertAll](/operation/insertall) is the exception — see `IsInsertAllBatchable` above. +- No table hints of any kind — `AreTableHintsSupported` is `false`; passing a non-null `hints` argument to any operation throws a `NotSupportedException`. +- No `TRUNCATE TABLE` statement (as of Vertica 5.0) — [Truncate](/operation/truncate) compiles to `DELETE FROM t` without a `WHERE` clause, which, unlike a real truncate, does not reset an `IDENTITY` column's next value. +- `MaxParameterCount` defaults to `1500`, lower than the `2098` most other providers default to. +- No native GUID property handler — `DbType.Guid` maps to Vertica's own `UUID` type, but there is no dedicated property handler equivalent to other providers' `GuidToByteArrayPropertyHandler` for drivers that need an explicit conversion. +- A second, near-duplicate property handler class (`RepoDb.PropertyHandlers.VerticaTimeToDateTimePropertyHandler`, as opposed to the documented `RepoDb.PropertyHandlers.Vertica.TimeToDateTimePropertyHandler`) exists in the source but is not referenced anywhere in the library or its tests — apparent leftover code from a refactor, not two intentionally-separate handlers. +- Only a standard Vertica server accessed via `Vertica.Data` is tested against. + +- Referenced the `RepoDb` package `v1.16.0`. +- Referenced the `Vertica.Data` package `v24.3.0`. diff --git a/pages/releases/verticabulk.md b/pages/releases/verticabulk.md new file mode 100644 index 0000000..8267b57 --- /dev/null +++ b/pages/releases/verticabulk.md @@ -0,0 +1,49 @@ +--- +layout: default +sidebar: releases +title: Vertica (Bulk) +description: "This page contains the latest information of the releases of RepoDb.Vertica.BulkOperations library." +permalink: /release/verticabulk +parent: RELEASES +--- + +# Releases for RepoDb.Vertica.BulkOperations + +--- + +View the NuGet package [here](https://www.nuget.org/packages/RepoDb.Vertica.BulkOperations) or download it directly [here](https://www.nuget.org/api/v2/package/RepoDb.Vertica.BulkOperations). + +## RepoDb.Vertica.BulkOperations (v0.0.1-alpha) - Preview + +Released: TBA + +New +{: .label .label-green } + +First release of the bulk operations extension for [RepoDb.Vertica](/release/vertica), built on `VerticaCopyStream` — `Vertica.Data`'s native `COPY ... FROM STDIN` streaming API. + +> **Verification status:** this package has been implemented and reviewed, but not yet exercised against a live Vertica instance. Verify the pseudo-table lifecycle, the `COPY` stream's value formatting (dates, booleans, binary), and the identity-read-back arithmetic end-to-end before relying on this package in production. + +### What's included + +- [BulkInsert](/operation/vertica/bulkinsert), [BulkMerge](/operation/vertica/bulkmerge), [BulkUpdate](/operation/vertica/bulkupdate), [BulkDelete](/operation/vertica/bulkdelete) and [BulkDeleteByKey](/operation/vertica/bulkdeletebykey), each with an `Async` overload, callable against a `VerticaConnection` with an entity list, a `DataTable`, or an `IDataReader` (`BulkDeleteByKey` takes a list of primary key values instead). +- An internal `COPY`-stream-based bulk-copy implementation, used to write into the target table (`BulkInsert` without `ReturnIdentity`) and into the pseudo (staging) table backing the other operations. It formats each row as a tab-delimited, newline-terminated record (backslash-escaping literal backslashes/tabs/CR/LF), with Postgres-lineage `t`/`f` boolean literals and hex-encoded binary values, and runs synchronously on the calling thread — the async overloads offload that synchronous sequence to a background thread instead, since `VerticaCopyStream` exposes no async API of its own. +- `VerticaBulkImportIdentityBehavior` (`KeepIdentity` default, `ReturnIdentity`) — controls whether identity values are sent as-is or read back. `ReturnIdentity` reads generated values back via a single `SELECT LAST_INSERT_ID()`, back-computing every row's individual identity from Vertica's contiguous, insertion-ordered `IDENTITY`/`AUTO_INCREMENT` assignment — no per-row round trip needed. +- `VerticaBulkImportPseudoTableType` (`Auto`, `Memory`, `Physical`) — selects the staging-table strategy backing `BulkMerge`, `BulkUpdate`, `BulkDelete`, `BulkDeleteByKey`, and `BulkInsert` (when `ReturnIdentity` is used). Every staging table is created with a per-call unique name, so `Memory` and `Physical` are both genuinely functional and safe under concurrent callers writing against the same target table. +- [VerticaBulkInsertMapItem](/class/vertica/verticabulkinsertmapitem) — explicit source-to-destination column mapping. Its optional `VerticaType` override is not currently consumed by the `COPY`-based implementation (Vertica's `COPY` parser infers wire format from the destination column's own server-side type) — kept only as a forward-looking escape hatch, matching every other bulk-operations package's map-item shape. +- [VerticaTraceKeys](/class/vertica/verticatracekeys) — the tracing key constants for the five bulk operations, for use with [ITrace](/interface/itrace). +- `BulkMerge` never generates a native `MERGE` statement (Vertica rejects it outright against a table with an `IDENTITY`/`AUTO_INCREMENT` column) — it always issues a separate `UPDATE ... FROM` followed by a separate `INSERT ... WHERE NOT EXISTS (...)`, as two distinct round trips rather than one compound statement. See [BulkMerge](/operation/vertica/bulkmerge#operation-sql-statements) for the full mechanics. + +### Known limitations (v1) + +- `BulkCopyTimeout` is accepted on every operation for signature symmetry with the other providers' bulk-operations packages, but `VerticaCopyStream` has no timeout-equivalent property to apply it to — the argument currently has no effect. +- Identity read-back (for both `BulkInsert` and `BulkMerge`) relies on `VerticaDbHelper.GetScopeIdentity`'s `SELECT LAST_INSERT_ID()` query plus a descending-offset back-computation that assumes Vertica assigns `IDENTITY`/`AUTO_INCREMENT` values strictly contiguously in insertion order. Neither the underlying query nor this assumption has been verified against a live Vertica instance; verify carefully under concurrent writers before relying on it in production. +- The `DbDataReader` overloads of `BulkInsert` and `BulkMerge` have no `identityBehavior` argument — a forward-only, single-pass reader cannot be rewound to correlate generated identity values back onto a source row. +- `BulkUpdate` (and the update half of `BulkMerge`) skip the generated `UPDATE` statement entirely when every staged field is also a qualifier (i.e. there is nothing left to actually update). +- The synchronous `WriteToServer`/`Execute` path is not thread-safe for concurrent use against the same connection — the pseudo table is created, indexed, read from, and dropped via other statements against that same connection immediately before/after the copy, so the copy itself must run on whatever thread is already driving that sequence. The async overloads offload to a background thread only because nothing else touches the connection concurrently while they await. +- Every bulk call against a table it hasn't seen before creates its own uniquely-named staging table (`CREATE TABLE`/`CREATE GLOBAL TEMPORARY TABLE`) and drops it once the call completes, rather than creating one per (table, pseudo table type) and reusing it across calls the way the Oracle bulk package does. +- This package inherits every [Known limitation of RepoDb.Vertica](/release/vertica) itself (e.g. no table hints, `MaxParameterCount` of `1500`, the unverified `Merge`/`MergeAll` compound-statement behavior). + +- Referenced the `RepoDb` package `v1.16.0`. +- Referenced the `RepoDb.Vertica` package `v0.0.1-alpha`. +- Referenced the `Vertica.Data` package `v24.3.0`.