go-reform / reform Goto Github PK

When I'm trying to define a composite Primary Key

//reform:raw_records
type RawRecord struct {
        ModelBase

        Date           time.Time `reform:"date,pk"`
        SensorId       int       `reform:"sensor_id,pk"`
        ChannelId      int       `reform:"channel_id,pk"`
        Value          int       `reform:"value"`
        ConvertedValue int       `reform:"converted_value"`
}

it returns an error:

reform: . RawRecord.go: reform: RawRecord has field SensorId with with duplicate "pk" label in "reform:" tag (first used by Date), it is not allowed

Seems the feature is not implemented, yet. But if I don't set any PK then reform thinks it's just a view (not a table). So I have to make an extra auto-increment field to use this model by reform.

This issue is just about "please implement support of composite Primary Keys" :)

Please help me to understand the reason of the next error:

$ go version
go version go1.5.1 linux/amd64
$ reform
panic: template: struct:27: illegal number syntax: "-"

goroutine 1 [running]:
text/template.Must(0x0, 0x7fafb582f028, 0xc82000aaf0, 0x0)
        /usr/lib/go/src/text/template/helper.go:23 +0x4b
main.init()
        /home/xaionaro/gocode/src/gopkg.in/reform.v1/reform/template.go:149 +0x212

$ cd /home/xaionaro/gocode/src/gopkg.in/reform.v1/reform; git log | head -1
commit d60dbfa976a39e47697f4038a37c7ae749030621

Usually, it's very useful to mark queries with some tags.

This code db.Tag("disable_user:%s", userId).Update(user) might produce the next query UPDATE /* disable_user:10 */ ...

3 steps:

1. Add dialect for MS SQL. That should be easy.
2. Fix reform and/or tests. Should be easy too, and likely not even required.
3. The hardest step: add Travis CI integration. MS SQL should be provisioned only on builder with specified target.

The first one who will send a working pull request will receive a squishable gopher from me and @GolangShow!

See #45 (comment)

Errors should be structures to have more context. Something like:

type ReformError struct {
  Op string
  Query string
  // args?
  Err error
}

func (re *ReformError) Error() string { }
func (re *ReformError) Cause() error { return re.Err }

Cause() is useful for https://godoc.org/github.com/pkg/errors#hdr-Retrieving_the_cause_of_an_error.

Why NewStruct return reform.Struct if we have our struct, which implement reform.Struct?

https://github.com/AlekSi/reform/blob/master/reform/template.go#L45

Try test local:

$ make test_mattn_go-sqlite3
rm -f reform-test.sqlite3
sqlite3 -bail reform-test.sqlite3 < internal/test/sql/sqlite3_init.sql
sqlite3 -bail reform-test.sqlite3 < internal/test/sql/data.sql
sqlite3 -bail reform-test.sqlite3 < internal/test/sql/sqlite3_set.sql
go test -coverprofile=test_mattn_go-sqlite3.cover
# reform
package reform_test
    imports github.com/AlekSi/reform/internal/test/models: use of internal package not allowed
FAIL    reform [setup failed]
make: *** [test_mattn_go-sqlite3] Error 1

Add method to TX to execute SAVEPOINT:

func (tx *TX) Begin() *TX { }

Commit() and Rollback() of nested "transaction" should execute RELEASE SAVEPOINT and ROLLBACK TO SAVEPOINT.

Relevant discussion: golang/go#7898.

Table & View need support schema prefix if need use it.
Example, we have same table in public schema & same table in chunks schema.
If we try change Name like this

func (v *personeView) Name() string {
    return "public.persone"
}

we have sql like a SELECT public.persone.id, public.persone.name, ...

tail needed for WHERE or LIMIT part of query. But function, whitch return just one struct need auto add limit 1 automatically.

Invoking an external gofmt command is relatively slow on Windows. Let's use go/format and do formatting ourselves when -gofmt=false. We should continue using gofmt -s by default because -s simplifies our generated code, and it is not possible yet to do it with go/format.

reform has different methods to make IN statements now

1. FindAllFrom(view View, column string, args ...interface{})
2. SelectRows(view View, tail string, args ...interface{})

The first one handles just one IN for a column, In the second I have to make placeholders for IN by myself.

What I want is the same logic as in the first case, but reform should automatically expand args element to $1, $2 if this element has a slice value.

• add AppVeyor configuration
• add badge

https://ci.appveyor.com/project/AlekSi/reform

See ba7795f. We don't really use it, so let's remove it for now.

For media_item it generates type media_itemTable and variable with the same name.

4 steps:

1. Decide on the drivers. The two most popular choices are https://github.com/mattn/go-oci8 and https://github.com/rana/ora. Are we going to support both? Which one is easier to install and test?
2. Add Drone CI configuration with Docker images from https://github.com/oracle/docker-images.
3. Add dialect for Oracle.
4. Fix code and/or tests.

Some experience with Oracle and Go drivers for it is pretty much required.

WIP branch: https://github.com/go-reform/reform/tree/custom-type

For example: https://github.com/satori/go.uuid

We need to add correct imports to generated files. Two ways to do it:

1. Import everything base file imports and use goimports (instead of gofmt) to remove unused ones.
2. Replace go/build with go/types and go/importer (maybe also golang.org/x/tools/go/packages).

See also #191.

• Add it to README.
• Enforce it (for example, during compilation with build tags).

Related to #23.

Several RDBMS support UPSERT (INSERT or UPDATE with a single statement):

• PostgreSQL 9.5+: docs, wiki.
• MySQL 5.5+: docs.
• SQLite3: docs.

We should add Upsert method to reform. It should use a single RDBMS-specific statement. Behaviour could be RDBMS-specific too but well documented (see PostgreSQL wiki link for the good overview). It should not try to emulate it in unsupported RDBMS with several queries and/or transactions, it will return an error instead.

WIP branch that could be useful: https://github.com/go-reform/reform/tree/upsert.

What I want for Insert or Update is to take default values from database if it's possible.

db.Update(obj, WithBlacklist("id", "created"))
db.Insert(obj, WithWhitelist("name", "surname"))

https://github.com/gocql/gocql

• https://golang.org/pkg/go/types/#AssignableTo
• https://golang.org/pkg/go/types/#ConvertibleTo

Those functions will help us generate code for #43 and #56.

Semi-crazy idea: linter to catch incorrect error handling. For example, some methods are documented to never return ErrNoRows. Linter can check user's code and find cases where result is compared with ErrNoRows.

Hell froze over! 😄

Let's add just enough for one-to-one and one-to-many relationships.

type Foo struct {
  ID int `reform:"id,pk"`
}

type Bar struct {
  ID int `reform:"id,pk"`
}

type Baz struct {
  FooID int   `reform:"foo_id"`
  BarID int   `reform:"bar_id"`
  Foo   *Foo  `reform:"FIXME_WHAT_THERE"`
  Bars  []Bar `reform:"FIXME_WHAT_THERE"`
}

LEFT OUTER JOINs only.

Currently, we run tests with 1.6 and tip for PostgreSQL 9.1 and SQLite.
We need to:

• add MySQL;
• add PostgreSQL 9.2-9.6;
• add different versions of MySQL.

Alternatively, we can switch to https://circleci.com – I was told it's better for such tests I don't think so anymore.

I would like to make use of Postgresql inheritance. To do that it's logical to use anonymous struct fields:

//reform:parents
type Parent struct {
    ID   int    `reform:"id,pk"`
    Name string `reform:"name"`
}

//reform:children
type Child struct {
    Parent
    Desc string `reform:"desc"`
} 

but that is not supported.

Help error

*main.Users

&main.Users{ID:1, Username:"zer", Password:"zer", Status:"active", Channel:"1", Created_at:1466744563, Last_activity_at:1466744563}panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x38 pc=0x8bffa]

goroutine 1 [running]:
panic(0x3681c0, 0xc82000a0c0)
        /usr/local/Cellar/go/1.6.2/libexec/src/runtime/panic.go:481 +0x3e6
github.com/go-reform/reform.(*Querier).Insert(0xc820015260, 0x6c28b0, 0xc82005e6c0, 0x0, 0x0)
        /Users/zer/golang/src/github.com/go-reform/reform/querier_commands.go:35 +0x70a
main.Conn()
        /Users/zer/dev/go/userapi/database.go:53 +0x7e6
main.main()
        /Users/zer/dev/go/userapi/main.go:22 +0x14
exit status 2

It will be nice if reform could map nested structs to a single table.
Example:

type DateTime struct {
    Date time.Time `reform: date`
    Timezone string `reform: timezone`
}

// reform:events
type Event struct {
    Start DateTime `reform: start_`
    End DateTime `reform: end_`
}

And that would map to a table with start_date, start_timezone, end_date and end_timezone columns.

At the moment, giving type Foo, we generate FooTable (or FooView). We can also generate FooSlice for a slice of Foo with some helpers:

type FooSlice []Foo

func (f FooSlice) Structs() []Struct { }
func MakeFooSlice(structs []Struct) FooSlice { }

This will give us helpful reflection-free methods:

• SelectAllTo
• something else?

I think it's a good idea for orm have a method for Insert array struct?

example

DB.Insert(s []Struct)

// SQL
INSERT INTO `table name` (`col1`, `col2`, `col3`) VALUES (s1.col1, s1.col2, s1.col3), (...)

If need, i can write It

A similar feature is offered by gorm

type String string

// reform:users
type User struct {
  B []byte `reform:"b"`
  S *String `reform:"s"`
}

Both fields cause reform to explode.

Maybe, just maybe.

    if s.q.Dialect != postgresql.Dialect {
        s.T().Skip("only PostgreSQL supports RETURNING syntax, other dialects support only integers from LastInsertId")
    }

This is no longer true.

Given all the limitations of prepared statements in Go – does anyone really need it? Please leave a comment.

WIP branch: https://github.com/go-reform/reform/tree/slice-pk

Valid use-case: UUID in binary form, probably should be used as []byte or [16]byte, not as string.

Remaining problems:

• HasPK implementation (operator != not defined on slice)
• SetPK implementation (cannot convert i64 (type int64) to type []uint8)

Related to #43.

Нужно как можно больше примеров. Хотя бы с инициализацией DB.

Столкнулся с проблемой при создании reform.NewDB, а именно с Logger.
Стандартный Logger я использовать не могу, Logger который interface тоже, а PrintfLogger, который сам напрашивается для использования, тоже не могу применить, тк не знаю кто поддерживает type Printf func(format string, a ...interface{})

Если бы пример на главной странице был более подробный, с открытием database/sql соединения, созданием логера и использовании — это очень сильно помогло бы разобраться что да как.

Some models can be updated with additional conditions, ie when I about to update OrderTable, I need to ensure the owner with user_id. For now I must do query first to ensure the owner.

What part of reform needs more documentation? Leave a comment.
Where will example help? Leave a comment.
Found a typo or grammar error? You can fix it right on GitHub.

We can use GitHub wiki, GitHub Pages, Hugo or GitBook for documentation. GORM docs is a good example. Another one – XORM.

We can also use https://asciinema.org for simple demo in README.

Update: see #135.

War. War with Travis CI (#5) never changes. While it's free and fast, it has a major downside: it's way too hard to add a non-trivial build configuration and debug it, and we want to test all the non-trivial configurations.

Let's move to self-hosted Drone CI 0.5. Two biggest benefits:

• It will make local testing (and CI setup debugging!) much easier: just drone exec.
• It will allow running tests with different versions of databases and Go much easier.

Downsides:

• I pay for hosting.
• A build is slow on a cheap virtual machine, and a fast VM is expensive.

My hosted instance is was here. Currently, it runs 60 configurations.

Maybe there will be free Drone service of OSS projects. Or maybe I should investigate how to spin up fast VM for build agent, make a build and shut it down.

AppVeyor will stay for native Windows builds. Travis CI will stay for current basic configurations because it's fast – at least until it starts to drag us down.

Steps:

• Full CI configuration.
• Local configuration for drone exec.
• Add codecov.io integration.
• Remove coveralls integration.
• Use gocoverutil.

reform-db init models should connect to a database, inspect information_schema or equivalent, and generate model files from it.

See also #75 and #102.

See PR #113.

3 steps:

1. Add dialect for Cassandra. That should be easy.
2. Fix reform and/or tests. Should be easy too, and likely not even required.
3. The hardest step: add Travis CI integration. Cassandra should be provisioned only on builder with specified target.

The first one who will send a working pull request will receive a squishable gopher from me and @GolangShow!

We need a unique logo. With a gopher. Some possible symbols:

• something database-related: UML database symbols, etc.
• something about reform: gopher on tribune?

@bosenok wanted to help.

• https://github.com/go-reform/golang-sql-benchmark
• https://github.com/go-reform/boilbench
• https://github.com/go-reform/orm-benchmark
• something else?
• small own benchmarks in this repo
• mention in README