Comments (13)
The odbc application bug was fixed in a fork https://github.com/arcusfelis/eodbc
odbc module became eodbc, and application name was also changed.
So, you can try to run with it.
From minuses: eodbc requires unixodbc to compile C code part.
While odbc is shipped with Erlang/OTP already compiled.
from mssql_ecto.
This is definitely a limitation of / bug in the erlang ODBC driver. We tried to support unsized character fields but the erlang ODBC driver mangles the return from them, so we were forced to restrict size to the maximum sized type, wvarchar(4000).
from mssql_ecto.
This should be the relevant issue for erlang-odbc.
https://bugs.erlang.org/browse/ERL-132
Is there a workaround for retrieving an nvarchar(max) RTF string or other technical solution? This is currently a showstopper for me. Thanks!
from mssql_ecto.
If the data is reliably under 4000 bytes a CAST(column AS nvarchar(4000))
on the select should work. If it's not I'm afraid you're out of luck until that issue in Erlang is resolved. I don't think the driver has any way of extracting large data that's working since the bug affects all variable length fields
We don't have any immediate plans to patch Erlang but if you'd like to give it a try I believe @toddharding from our team had given it a shot and identified at least the broad strokes of what the problem is.
from mssql_ecto.
Thank you for the quick reply. Unfortunately that doesn't work for me. Those RTF strings are huge!
from mssql_ecto.
Nice work! We'll upgrade and hopefully the unixodbc dependency isn't too much trouble for people.
from mssql_ecto.
Has this been considered as a possible solution?
http://erlang-odbc-tips.colefichter.ca/
See "Support for NVARCHAR(MAX)" section.
from mssql_ecto.
from mssql_ecto.
Thanks @arcusfelis
How would I go about installing and using your eodbc fork (debian system)?
And I assume I'd have to fork mssql_ecto to use eodbc instead of erlang-odbc?
from mssql_ecto.
-
You would need to install unixodbc build deps (basically, what you need to install to build Erlang with odbc).
-
Add eodbc as a dep into rebar.config.
-
Use eodbc module instead of odbc.
from mssql_ecto.
Thanks @arcusfelis
I'm still not clear how I could get eodbc up and running with mssql_ecto / mssqlex.
I've forked mssqlex and replaced :odbc
with :eodbc
where (I beleive) appropriate.
Additionally, I'm installing eodbc in my Pheonix app as follows:
defp deps do
[
{:phoenix, "~> 1.3.4"},
{:phoenix_pubsub, "~> 1.1.0"},
{:phoenix_ecto, "~> 3.4.0"},
{:mssql_ecto, git: "git://github.com/cpursley/mssql_ecto.git", branch: "c/eodbc", override: true},
...
Any other suggestions?
from mssql_ecto.
Ok, I was able to get eodbc
loaded by installing it via mix and swapping "odbc" for "eodbc" where appropriate in the mssqlex library. That approach works and it's pretty handy that you can install erlang rebar libraries easily via hex.
However, there are some esoteric errors coming up when running queries where the column type is either varchar(max) or nvarchar(max).
In iex:
{:ok, conn} = Mssqlex.ODBC.init(conn_str)
This works for nvarchar(max). However, the columns are returned as binary.
> :eodbc.sql_query(conn, 'SELECT * FROM dbo.tblSomeTable AS st WHERE st.ID = 1')
=> {:selected,
['ID', 'SomeColumn'],
[[1, <<72, 0, 52, 0, 115, 0, 73, 0, 65, 0, 65, 0, 65, 0, 65, 0, 65, 0, 65, 0, 65, 0, 65, 0, 65, 0, 50, 0, 49, 0, 89, 0, 87, ...>>]
]}
This does not work when varchar(max):
> :eodbc.sql_query(conn, 'SELECT * FROM dbo.tblSomeTable AS st WHERE st.ID = 1')
=> [error] GenServer #PID<0.472.0> terminating
** (stop) {:port_exit, :killed}
Last message: {#Port<0.102>, {:exit_status, 139}}
State: {:state, #Port<0.102>, {#PID<0.389.0>, #Reference<0.425679474.514588673.206201>}, #PID<0.389.0>, :undefined, :off, true, true, :on, :connected, :undefined, 0, [#Port<0.100>, #Port<0.101>], #Port<0.103>, #Port<0.104>}
And via Ecto/Mssqlex, neither varchar(max) nor nvarchar(max) work.
Error for nvarchar(max):
> SomeTable |> Repo.get(1)
=> [error] [79, 68, 66, 67, 58, 32, 114, 101, 99, 101, 105, 118, 101, 100, 32, 117, 110, 101, 120, 112, 101, 99, 116, 101, 100, 32, 105, 110, 102, 111, 58, 32, [123, ['tcp_closed', 44, '#Port<0.62>'], 125], 10]
[error] GenServer #PID<0.378.0> terminating
** (stop) {:port_exit, :could_not_bind_data_buffers}
Last message: {#Port<0.58>, {:exit_status, 22}}
State: {:state, #Port<0.58>, {#PID<0.359.0>, #Reference<0.1246316755.1851523073.245020>}, #PID<0.359.0>, :undefined, :off, true, true, :on, :connected, :undefined, 0, [#Port<0.54>, #Port<0.55>], #Port<0.61>, #Port<0.62>}
- The
:could_not_bind_data_buffers
error appears to be similar to this issue: ClickHouse/clickhouse-odbc#27 - The corresponding exit code in eobcd: https://github.com/arcusfelis/eodbc/blob/748d613846731c3d776c8f22600d8b6d89c26435/src/eodbc_internal.hrl#L161
Error for varchar(max):
SomeTable |> Repo.get(1)
[error] [79, 68, 66, 67, 58, 32, 114, 101, 99, 101, 105, 118, 101, 100, 32, 117, 110, 101, 120, 112, 101, 99, 116, 101, 100, 32, 105, 110, 102, 111, 58, 32, [123, ['tcp_closed', 44, '#Port<0.57>'], 125], 10]
[error] GenServer #PID<0.374.0> terminating
** (stop) {:port_exit, :killed}
Last message: {#Port<0.53>, {:exit_status, 139}}
State: {:state, #Port<0.53>, {#PID<0.358.0>, #Reference<0.2075734545.4001366017.31751>}, #PID<0.358.0>, :undefined, :off, true, true, :on, :connected, :undefined, 0, [#Port<0.49>, #Port<0.50>], #Port<0.56>, #Port<0.57>}
It looks like there's two things that need to happen to get at least nvarchar(max) working:
- Get odbc:param_query to submitt params correclty
- Decoding binary type (is this an erlang encoding?)
Does this sound right @arcusfelis & @jbachhardie?
from mssql_ecto.
from mssql_ecto.
Related Issues (20)
- support for xml column type HOT 4
- Variable insertion misaligned with using WHERE IN HOT 4
- Run tests using latest version of Elixir
- Support for uniqueidentifier / UUIDs HOT 5
- Installing Erlang ODBC on Windows 2012 R2 HOT 9
- Encoding issues with nvarchar(max) columns? HOT 1
- Support Ecto 2.2 HOT 7
- App created with phx.new test --database mssql won't compile HOT 6
- DML statement cannot have any enabled triggers if the statement contains an OUTPUT clause without INTO clause. HOT 10
- Compilation error in file lib/mssql_ecto/query_string.ex == HOT 1
- How to handle capital letters in database column HOT 1
- UUID ** (ArgumentError) invalid or unknown type :uuid for field :DemoCategoryKey HOT 2
- Column type not supported | ODBC_CODE | SQL_SERVER_CODE 0 using type uniqueidentifier HOT 1
- Difficulty setting up HOT 9
- Can't open lib HOT 1
- Concurrent DB Tests Failing HOT 1
- Alter table remove column migration removes primary key
- Associations do not work with non-standard primary keys HOT 3
- Preloading associations with differing schemas fails
- Ecto-3 support. HOT 18
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mssql_ecto.