Documentation
¶
Overview ¶
Package ds (short for "data store") is a key-value store with hash indexes. It allows for rudimentary but lightning fast retrieval of grouped or relevant data without having to iterate over all objects in the store.
Define the primary key, indexed keys, and unique keys as tags on struct fields, and DS takes care of the rest.
Index ¶
Examples ¶
Constants ¶
const ( ErrMissingRequiredValue = "missing required value" ErrPropertyChanged = "property changed" ErrDuplicatePrimaryKey = "duplicate value for primary key" ErrDuplicateUnique = "duplicate value for unique field" ErrFieldNotIndexed = "field not indexed" ErrFieldNotUnique = "field not unique" ErrMigrateTablePathNotFound = "table path" ErrBadTableFile = "bad table file" )
Error constant
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
Fields []Field
TypeOf string
PrimaryKey string
Indexes []string
Uniques []string
LastInsertIndex uint64
Version int
}
Config describes ds table configuration
type GetOptions ¶
type GetOptions struct {
// Should the results be sorted. Does nothing for unsorted tables
Sorted bool
// If results are to be sorted, should they be from most recent to oldest (true) or invese (false)
Ascending bool
// The maximum number of entries to return. 0 means unlimited
Max int
}
GetOptions describes options for getting entries from a DS table
type IReadTransaction ¶ added in v1.8.0
type IReadTransaction[T any] interface { // Get will get a single entry by its primary key. Returns (nil, nil) if nothing found. Get(primaryKey any) (*T, error) // GetIndex will get multiple entries that contain the same value for the specified indexed field. // Result is not ordered. Use GetIndexSorted to return a sorted slice. // Returns an empty array if nothing found. GetIndex(fieldName string, value any, options *GetOptions) ([]*T, error) // GetUnique will get a single entry based on the value of the provided unique field. // Returns (nil, nil) if nothing found. GetUnique(fieldName string, value any) (*T, error) // GetAll will get all of the entries in the table. GetAll(options *GetOptions) ([]*T, error) }
IReadTransaction describes an interface for performing read-only operations on a table.
type IReadWriteTransaction ¶ added in v1.8.0
type IReadWriteTransaction[T any] interface { IReadTransaction[T] // Add will add a new object to the table. o must the the same type that was used to register the table and cannot be a pointer. Add(o T) error // Delete will delete the provided object and clean indexes Delete(o T) error // DeletePrimaryKey will delete the object with the associated primary key and clean indexes. Does nothing if not object // matches the given primary key. DeletePrimaryKey(o any) error // DeleteUnique will delete the object with the associated unique value and clean indexes. Does nothing if no object // matched the given unique fields value. DeleteUnique(field string, o any) error // DeleteAllIndex will delete all objects matching the given indexed fields value DeleteAllIndex(fieldName string, value any) error // DeleteAll delete all objects from the table DeleteAll() error // Update will update an existing object in the table. The primary key must match for this object // otherwise it will just be inserted as a new object. Updated objects do not change positions in a sorted // table. Update(o T) error }
IReadWriteTransaction describes an interface for performing read or write operations on a table.
type MigrateParams ¶
type MigrateParams[OldType any, NewType any] struct { // TablePath the path to the existing table file TablePath string // NewPath (optional) is the path for the new table file. If blank then the existing table path is re-used. NewPath string // DisableSorting (optional) if the current table is sorted, set this to true to disable sorting // Note: This is irreversible! DisableSorting bool // MigrateObject method called for each entry in the table in reverse order. Return a new type, error, or nil. // Migration is halted if an error is returned. // Return (nil, nil) and the entry will be skipped from migration, but migration will continue. MigrateObject func(o *OldType) (*NewType, error) // KeepBackup (optional) if false the backup copy of the table will be discarded if the migration was successful. If // true, the copy is not deleted. KeepBackup bool }
MigrateParams describes the parameters to perform a DS table migration. All fields are required unless otherwise specified.
- `OldType` is an instance of a struct object that has the same definition as the existing table.
- `NewType` is an instance of a struct object that has the definition that shall be used. This can be the same as the OldType.
type MigrationResults ¶
type MigrationResults struct {
// Success was the migration successful
Success bool
// Error if unsuccessful, this will be the error that caused the failure
Error error
// EntriesMigrated the number of entries migrated
EntriesMigrated uint
// EntriesSkipped the number of entries skipped
EntriesSkipped uint
}
MigrationResults describes results from a migration
func Migrate ¶
func Migrate[OldType any, NewType any](params MigrateParams[OldType, NewType]) (results MigrationResults)
Migrate will migrate a DS table from one object type to another. You must migrate if the old data type is not compatible with the new type, such as if an existing field was changed. You don't need to migrate if you add a new field or remove an existing field.
Before the existing data is touched, a copy is made with "_backup" appended to the filename, and a new table file is created with the migrated entries. Upon successful migration, the backup copy is deleted by default. If the table being migrated is sorted, the original order is preserved.
Ensure you read the documentation of the MigrateParams struct, as it goes into greater detail on the parameters required for migration, and what they do.
Example ¶
package main
import (
"fmt"
"github.com/ecnepsnai/ds"
)
func main() {
// Define a struct that maps to the current type used in the table
type oldType struct {
Username string `ds:"primary"`
Email string `ds:"unique"`
FirstName string
LastName string
}
// Define your new struct
type newType struct {
Username string `ds:"primary"`
Email string `ds:"unique"`
Name string
}
// In this example, we're merging the "FirstName" and "LastName" fields of the User object to
// just a single "Name" field
// NewType can be the same as the old type if you aren't changing the struct
result := ds.Migrate(ds.MigrateParams[oldType, newType]{
TablePath: "/path/to/table.db",
NewPath: "/path/to/table.db", // You can specify the same path, or a new one if you want
MigrateObject: func(old *oldType) (*newType, error) {
// Within the MigrateObject function you can:
// 1. Return a object of the NewType (specified in the MigrateParams)
// 2. Return an error and the migration will abort
// 3. Return nil and this entry will be skipped
return &newType{
Username: old.Username,
Email: old.Email,
Name: old.FirstName + " " + old.LastName,
}, nil
},
})
if !result.Success {
// Migration failed.
panic(result.Error)
}
fmt.Printf("Migration successful. Entries migrated: %d, skipped: %d\n", result.EntriesMigrated, result.EntriesSkipped)
}
type Options ¶
type Options struct {
// DisableSorting disable all sorting features. This will make tables smaller, and inserts/removes/deletes faster.
DisableSorting bool
// contains filtered or unexported fields
}
Options describes options for DS tables. Once set, these cannot be changed.
type Table ¶
Table describes a ds table. A table is mapped to a single registered object type and contains both the data and the indexes.
func Register ¶
Register will register an instance of a struct with ds, creating a table (or opening an existing table) for this type at the specified file path.
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
// Primary fields represent the primary key of the object. Your object must have exactly one primary field
// and its value is unique
Username string `ds:"primary"`
// Unique fields function just like primary fields except any field (other than the primary field) can be unique
Email string `ds:"unique"`
// Index fields represent fields where objects with identical values are grouped together so they can be fetched
// quickly later
Enabled bool `ds:"index"`
// Fields with no ds tag are saved, but you can't fetch based on their value, and can have duplicate values
// between entries
Password string
}
tablePath := "user.db"
table, err := ds.Register[User](tablePath, nil)
if err != nil {
panic(err)
}
// Don't forget to close your table when you're finished
table.Close()
}
func (*Table[T]) Close ¶
func (table *Table[T]) Close()
Close will close the table. This will not panic if the table has not been opened or already been closed.
func (*Table[T]) IsIndexed ¶
IsIndexed is the given field indexed
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Email string `ds:"email"`
Enabled bool `ds:"index"`
}
tablePath := "user.db"
table, err := ds.Register[User](tablePath, nil)
if err != nil {
panic(err)
}
table.IsIndexed("Username") // returns False
table.IsIndexed("Enabled") // returns True
}
func (*Table[T]) IsUnique ¶
IsUnique is the given field unique
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Email string `ds:"email"`
Enabled bool `ds:"index"`
}
tablePath := "user.db"
table, err := ds.Register[User](tablePath, nil)
if err != nil {
panic(err)
}
table.IsUnique("Username") // returns False
table.IsUnique("Email") // returns True
}
func (*Table[T]) StartRead ¶ added in v1.8.0
func (table *Table[T]) StartRead(ctx func(tx IReadTransaction[T]) error) error
StartRead will start a new read-only transaction on the table. ctx will be called when the transaction is ready. This method may block if there is an active write transaction.
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Password string
Email string `ds:"unique"`
Enabled bool `ds:"index"`
}
var table *ds.Table[User] // Assumes the table is already registered, see ds.Register for an example
// Get the user with username "example"
var user *User
table.StartRead(func(tx ds.IReadTransaction[User]) (err error) {
user, err = tx.Get("example")
if err != nil {
return err // error fetching data
}
if user == nil {
return nil // no user found
}
return
})
if user == nil {
panic("No user found!")
}
// Use the user object
}
func (*Table[T]) StartWrite ¶ added in v1.8.0
func (table *Table[T]) StartWrite(ctx func(tx IReadWriteTransaction[T]) error) error
StartWrite will start a new read-write transaction on the table. ctx will be called when the transaction is ready. This method may block if there is an active transaction.
Note that any any errors or partial changes made during ctx are not reverted.
Example ¶
package main
import (
"github.com/ecnepsnai/ds"
)
func main() {
type User struct {
Username string `ds:"primary"`
Password string
Email string `ds:"unique"`
Enabled bool `ds:"index"`
}
var table *ds.Table[User] // Assumes the table is already registered, see ds.Register for an example
newUser := User{
Username: "ian",
Password: "hunter2",
Email: "email@domain",
Enabled: true,
}
err := table.StartWrite(func(tx ds.IReadWriteTransaction[User]) error {
return tx.Add(newUser)
})
if err != nil {
panic(err)
}
}