Comments (7)
pinging @alexander-beedie here.
from polars.
@alexander-beedie , potentially simpler repro / root-cause. I know it is not the same error, but kept trying to reduce the issue and got here. If you think it is a separate issue, I will create one
import polars as pl
pl.__version__ # 1.0.0
users = pl.DataFrame({"id": "1"})
user_groups = pl.DataFrame({"user_id": "1"})
pl.sql("""
WITH user_by_email AS (SELECT id FROM users)
SELECT user_groups.user_id AS input_user_id
FROM user_groups
INNER JOIN user_by_email ON user_groups.user_id = user_by_email.id
-- users <--- if you remove this line, you will get an error
-- It is commented out, but it must have to do with global variable discovery
"""
).collect()
# Raises SQLInterfaceError: relation 'users' was not found when `-- users` line is removed
EDIT: separate issue, I will raise one, please ignore
from polars.
Tidier repro
# removing the email field fixes it
users = pl.DataFrame({"id": "1", "email": "abc"})
user_groups = pl.DataFrame({"id": "1"})
group_group = pl.DataFrame({"id": "1"})
deals = pl.DataFrame({"id": "1"})
ctx = pl.SQLContext({
"users": users,
"user_groups": user_groups,
"group_group": group_group,
"deals": deals,
})
ctx.execute("""
WITH user_by_email AS (SELECT id FROM users),
user_child AS (
SELECT group_group.id
FROM user_groups
-- removing this join fixes it
INNER JOIN user_by_email ON user_groups.id = user_by_email.id
INNER JOIN group_group ON user_groups.id = group_group.id
)
SELECT *
FROM deals
WHERE (
deals.id IN (
SELECT id
FROM users
-- removing the below line changes the error from
-- `PanicException: internal error: entered unreachable code` to
-- `PanicException: called `Option::unwrap()` on a `None` value`
WHERE id = '1'
)
OR deals.id IN (
SELECT DISTINCT user_groups.id
FROM user_groups AS left
-- removing this join also fixes it
INNER JOIN user_by_email ON user_groups.id = user_by_email.id
)
OR deals.id IN (
SELECT DISTINCT user_groups.id
FROM user_groups
-- removing this join also fixes it
INNER JOIN user_child ON user_groups.id = user_child.id
)
)
""",
eager=True
)
# PanicException: internal error: entered unreachable code
And have been still able to get a panic (thought not the same "entered unreachable code" error) with a smaller repro again
a = pl.DataFrame({"id": "1"})
ctx = pl.SQLContext({"a": a})
ctx.execute("""
-- changing this from `SELECT a.id` to `SELECT id` fixes it
-- even though this CTE is never used
-- removing the CTE also fixes the issue
WITH c AS (SELECT a.id FROM a)
SELECT *
FROM a
WHERE id IN (
SELECT id
FROM a
-- removing this join fixes the issue
INNER JOIN a AS a2 ON a.id = a2.id
)
""",
eager=True
)
# PanicException: called `Option::unwrap()` on a `None` value
from polars.
Oh yeah.. enjoy!
from polars.
I did reproduce the unreachable code error, but it happens elsewhere so not entirely sure if it is the same underlying issue.
users = groups = deals = pl.DataFrame({"id": [1]})
pl.sql("""
with
A as ( select id from groups join users using (id) )
select * from deals
where
id in ( select id from A join A as B using (id) )
""").collect()
# thread '<unnamed>' panicked at crates/polars-plan/src/plans/ir/schema.rs:106:24:
# internal error: entered unreachable code
from polars.
@alexander-beedie I see that in the example above by @cmdlineluser it creates an external context. I really suspect this to be the culprit. Can we use horizontal concat here instead?
from polars.
@alexander-beedie I see that in the example above by @cmdlineluser it creates an external context. I really suspect this to be the culprit. Can we use horizontal concat here instead?
I'll find out (though I'm on vacation at the moment so can't dig in properly yet - am only able to sneak the time for small PRs at the moment 😉)
from polars.
Related Issues (20)
- QOL improvements for .rolling
- Include example with function accepting multiple arguments in `Expr.map_batches`
- Documentation issue in `normalize`/`name` parameter from `.value_counts()` method HOT 2
- writing to os.devnull
- `Series[list].explode()` should not return `None` for empty lists HOT 2
- write_database to snowflake with adbc engine spouts context canceled error log
- Cannot tell if hvplot version 0.10.0 >= 0.9.1 HOT 2
- dtype 'Time' gets converted to i64 when collect(streaming=True) is used. HOT 1
- `.agg_groups()` PanicException when not used in a group_by context
- Additional Parameter for json_normalize HOT 5
- fold shouldn't require that acc and exprs share the same dtype
- Adding `descending` parameter to `Expr.over` HOT 5
- polars.LazyFrame.head recommends using fetch() HOT 1
- Reading large json file error: ComputeError: InputTooLarge at character 0
- Serialize for AnyType has a todo!() HOT 1
- File cache invalidation not triggered for HTTP if size is the same
- Loading parquet written from an Arrow table produces non-deterministic incorrect numbers since 1.2.0 HOT 3
- illegal hardware instruction with python 3.12.4 and polars 1.1.0 on MacOS Sanoma 14, M2 HOT 4
- Panic when call `hash()` on `struct` dtype HOT 2
- Multiple - Reading into a single DataFrame - read_csv - Error when using encoding = latin1 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polars.