Originally submitted by: Marti Raudsepp

We're using lots of numeric/Decimal types in our database, with a Django/mod_wsgi/Apache frontend, using WSGI daemon mode. Sometimes we spuriously receive "Can't adapt type 'Decimal'" errors for some of the front-end processes, from queries that involve converting Python Decimal literals to PostgreSQL numerics. Unfortunately I have not found a consistent way to reproduce this.

When a process goes into this state, it seems to remain in it until restarted. Whether it happens straight after launching a process, or develops this problem while running, I also do not know.

There's also this mailing list thread of someone with a similar problem: http://www.mail-archive.com/[email protected]/msg103365.html
And related blog entry: http://www.defitek.com/blog/2010/06/29/cant-adapt-type-decimal-error-with-django-apache-postgresql-psycopg2/
His problem was solved by separating two sites into individual daemon process groups, but that's a workaround rather than a fix.

However, I only have 1 Django application running on my server. I'm also using a single WSGI thread per process:
WSGIDaemonProcess xyz processes=32 threads=1 display-name=%{GROUP}

Although this means there's only 1 request handler thread per process, mod_wsgi creates 2 other threads presumably for internal statekeeping.

I will continue looking for details on this bug.

Originally submitted by: Jean-Baptiste Quenot

Originally submitted as number 14 - http://psycopg.lighthouseapp.com/projects/62710/tickets/14

Patch at http://dl.dropbox.com/u/5836407/Patches/Psycopg2/0004-Use-pthread_self-instead-of-getpid.patch

Helps to identify the active thread in the debug logs

It looks like copy_from does not stop reading after an error. When the input file is short, it is okay. But when the input file is very long, it is really boring to wait for the entire file to be read just to discover there is an error on the 10th row.

Given the same input file, it looks like psql \copy command behaves correctly and stops just after the incorrect row, without reading the entire file. I have checked that just by looking at the command execution time that seems proportional to the number of processed rows.

Here is a script to reproduce this bug (just create a database "test" and run the script):

https://gist.github.com/805669

Originally submitted by: Mark Lakewood

Hi all.

Im running a raw sql query through django 1.2.5 that uses the to_timestamp() postgres function. in psycopg2 2.0.x it works. when I use 2.4 it puts out the following traceback

http://pgsql.privatepaste.com/3006a9e813

Originally submitted by: Pumukel

Hello,

might it be possible to release a seperate egg of ZPsycopgDA on pypi, with dependency for the accurat psycopg2 egg?

For Zope and Plone users that would be a huge advantage, so it could be used directly in buildout.

Thanking you in anticipation

Originally submitted by: Nicole

You may use tags such as rel-2.2.1 or os-linux to help us categorize the bug.

This is a test ticket. Daniele should be notified but the other account member should not.

Originally submitted by: tamagotchi

Hi,

I have been using psycopg2 on PostgreSQL 8.3, 8.4 on Linux and Windows without any issues.
Now I started working on a Mac OSX machine and installed the latest PostgreSQL 9 + psycopgs2 2.4 (dt dec pq3 ext) + Python 2.7.1
And I noticed that my application (which is using SQLAlchemy) can't write into bytea fields anymore.
I'm getting:
TypeError: can't escape unicode to binary

I boiled it down to this simple code block:

b = psycopg2.Binary(u"XYZABCTEst")
c = psycopg2.connect("dbname=acttest")
cur = c.cursor()
cur.execute("INSERT INTO users (userid, preferences) VALUES ('userid', %(prefs)s) RETURNING users.id", {'prefs': b})

If I use a regular string
b = psycopg2.Binary("XYZABCTEst")
everything works fine.

Binary is supposed to work with unicode objects, isn't it?

Thanks,

-T

Using named cursor, records are fetched one at time, with a lot of network roundtrips. A more efficient way of fetching data is using fetchmany.

There may be a default number of record to be fetched internally to make iteration more efficient.

Originally submitted by: Will Duncan

Originally submitted as number 25 - http://psycopg.lighthouseapp.com/projects/62710/tickets/25

You may use tags such as rel-2.2.1 or os-linux to help us categorize the bug.

Originally submitted by: Kevin

Hi guys, thanks for fixing the error with connecting to a pooler. It fixed pgpool as well. I meant to submit a bug on that before. My issue seems to be that there are intermittent times when I'm unable to write via cursor.execute when connecting via pgpool. (I thought it was working earlier today when I made this bare code..?) I can't replicate the issue at all when connecting directly. I can see that it is matching the code to quit but it seems like nothing is getting sent to the server, as verified in the postgres logs. I can create a new connection via pgpool and that works fine. Is there something I'm doing wrong? Also at this point, I'm unable to control-c out of the program or kill it. Here's my simplified code:

!/usr/bin/python26

import select
import psycopg2
import psycopg2.extensions

conn = psycopg2.connect("dbname=jobs user=postgres host=localhost port=9999")
conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)

curs = conn.cursor()
curs.execute("LISTEN test; LISTEN test_quit;")

while 1:
if select.select([conn],[],[],5) == ([],[],[]):
print ""
else:
try:
conn.poll()

            while conn.notifies:
                    noti = conn.notifies.pop(0)
                    print "Got NOTIFY:", noti

                    if noti[1] == "test":
                            print "test"
                    elif noti[1] == "test_quit":
                            print "QUIT"
                            curs.execute("UNLISTEN test; UNLISTEN test_quit;")
                            curs.close();
                            conn.close();
     except Exception, data:
            print data
            pass

Originally submitted by: Giuseppe Tofoni

Good Morning

I have the following problem with psycopg2-2.3.2:

python 2.6.4 (r264:75706, Dec 5 2009, 21:18:28)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import psycopg2
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.6/site-packages/psycopg2/init.py", line 69, in
from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: libpq.so.5: cannot open shared object file: No such file or directory

My dir postgresql (vers. 9.0.3)
/usr/local/pgsql

My setup.cfg
pg_config=/usr/local/pgsql/bin/pg_config

there were no errors in installation.

I linked /usr/local/pgsql/libpq.so.5 in /usr/lib and everything is fine.

Is that correct?

Thanks for you help and advice.

Originally submitted as number 3 - http://psycopg.lighthouseapp.com/projects/62710/tickets/3

Add support for the notification payload available in PG 9.0

Under development in piro's notify-payload branch.

Originally submitted by: Jeff Kowalczyk

Originally submitted as number 18 - http://psycopg.lighthouseapp.com/projects/62710/tickets/18

When I try to build psycopg2 with a pip -e /path/to/psycopg2, I get the following error:

psycopg/connection_type.c:810: error: 'psyco_conn_get_exception' undeclared here (not in a function)

error: command 'gcc' failed with exit status 1

This error is consistent for psycopg versions 2.2.2, 2.3.0_beta1, and git.

Using the alternative install method works fine, I'm using a virtualenv, BTW:

(cd /path/to/psycopg2 && python setup.py install)

To keep the environment reproducible on multiple platforms, the virtualenvs use non-system pythons, e.g. an /opt/python2x. These are built with zc-buildout 1.4.4, zc.cmmi recipe:

# cat /opt/python27/buildout.cfg 
[buildout]
parts = python
parts-directory = .
download-cache = cache

[python]
recipe = zc.recipe.cmmi
url = http://www.python.org/ftp/python/2.7.1/Python-2.7.1rc1.tgz
extra_options = --enable-unicode=ucs2

The complete pip log is below:

------------------------------------------------------------
/path/to/acme/bin/pip run on Wed Nov 17 11:06:48 2010
Obtaining file:///path/to/psycopg2
  Running setup.py egg_info for package from file:///path/to/psycopg2
    running egg_info
    writing psycopg2.egg-info/PKG-INFO
    writing top-level names to psycopg2.egg-info/top_level.txt
    writing dependency_links to psycopg2.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'psycopg2.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.html' under directory 'doc'
    warning: no files found matching '*.js' under directory 'doc'
    warning: no files found matching '*' under directory 'doc/html'
    no previously-included directories found matching 'doc/src/_build'
    warning: no files found matching 'PKG-INFO'
    warning: no files found matching 'MANIFEST'
    writing manifest file 'psycopg2.egg-info/SOURCES.txt'
Installing collected packages: psycopg2
  Running setup.py develop for psycopg2
    Running command /path/to/acme/bin/python -c "import setuptools; __file__='/path/to/psycopg2/setup.py'; execfile('/path/to/psycopg2/setup.py')" develop --no-deps
    running develop
    install_dir /path/to/acme/lib/python2.7/site-packages/
    running egg_info
    writing psycopg2.egg-info/PKG-INFO
    writing top-level names to psycopg2.egg-info/top_level.txt
    writing dependency_links to psycopg2.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'psycopg2.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.html' under directory 'doc'
    warning: no files found matching '*.js' under directory 'doc'
    warning: no files found matching '*' under directory 'doc/html'
    no previously-included directories found matching 'doc/src/_build'
    warning: no files found matching 'PKG-INFO'
    warning: no files found matching 'MANIFEST'
    writing manifest file 'psycopg2.egg-info/SOURCES.txt'
    running build_ext
    building 'psycopg2._psycopg' extension
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement -Wdeclaration-after-statement
    /opt/python27/python/include/python2.7/datetime.h:188: warning: ‘PyDateTimeAPI’ defined but not used
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/pqpath.c -o build/temp.linux-x86_64-2.7/psycopg/pqpath.o -Wdeclaration-after-statement -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/typecast.c -o build/temp.linux-x86_64-2.7/psycopg/typecast.o -Wdeclaration-after-statement -Wdeclaration-after-statement
    psycopg/typecast.c:40: warning: ‘skip_until_space’ defined but not used
    /opt/python27/python/include/python2.7/datetime.h:188: warning: ‘PyDateTimeAPI’ defined but not used
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/microprotocols.c -o build/temp.linux-x86_64-2.7/psycopg/microprotocols.o -Wdeclaration-after-statement -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/microprotocols_proto.c -o build/temp.linux-x86_64-2.7/psycopg/microprotocols_proto.o -Wdeclaration-after-statement -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/connection_type.c -o build/temp.linux-x86_64-2.7/psycopg/connection_type.o -Wdeclaration-after-statement -Wdeclaration-after-statement
    psycopg/connection_type.c:796: error: ‘psyco_conn_get_exception’ undeclared here (not in a function)
    error: command 'gcc' failed with exit status 1
    Complete output from command /path/to/acme/bin/python -c "import setuptools; __file__='/path/to/psycopg2/setup.py'; execfile('/path/to/psycopg2/setup.py')" develop --no-deps:
    running develop

install_dir /path/to/acme/lib/python2.7/site-packages/

running egg_info

writing psycopg2.egg-info/PKG-INFO

writing top-level names to psycopg2.egg-info/top_level.txt

writing dependency_links to psycopg2.egg-info/dependency_links.txt

warning: manifest_maker: standard file '-c' not found



reading manifest file 'psycopg2.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

warning: no files found matching '*.html' under directory 'doc'

warning: no files found matching '*.js' under directory 'doc'

warning: no files found matching '*' under directory 'doc/html'

no previously-included directories found matching 'doc/src/_build'

warning: no files found matching 'PKG-INFO'

warning: no files found matching 'MANIFEST'

writing manifest file 'psycopg2.egg-info/SOURCES.txt'

running build_ext

building 'psycopg2._psycopg' extension

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement -Wdeclaration-after-statement

/opt/python27/python/include/python2.7/datetime.h:188: warning: ‘PyDateTimeAPI’ defined but not used

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/pqpath.c -o build/temp.linux-x86_64-2.7/psycopg/pqpath.o -Wdeclaration-after-statement -Wdeclaration-after-statement

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/typecast.c -o build/temp.linux-x86_64-2.7/psycopg/typecast.o -Wdeclaration-after-statement -Wdeclaration-after-statement

psycopg/typecast.c:40: warning: ‘skip_until_space’ defined but not used

/opt/python27/python/include/python2.7/datetime.h:188: warning: ‘PyDateTimeAPI’ defined but not used

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/microprotocols.c -o build/temp.linux-x86_64-2.7/psycopg/microprotocols.o -Wdeclaration-after-statement -Wdeclaration-after-statement

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/microprotocols_proto.c -o build/temp.linux-x86_64-2.7/psycopg/microprotocols_proto.o -Wdeclaration-after-statement -Wdeclaration-after-statement

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.3.0-beta2 (dt dec ext pq3)" -DPG_VERSION_HEX=0x090001 -DPG_VERSION_HEX=0x090001 -I/opt/python27/python/include/python2.7 -I. -I/usr/include/postgresql-9.0 -I/usr/include/postgresql-9.0/server -c psycopg/connection_type.c -o build/temp.linux-x86_64-2.7/psycopg/connection_type.o -Wdeclaration-after-statement -Wdeclaration-after-statement

psycopg/connection_type.c:796: error: ‘psyco_conn_get_exception’ undeclared here (not in a function)

error: command 'gcc' failed with exit status 1

----------------------------------------
Command /path/to/acme/bin/python -c "import setuptools; __file__='/path/to/psycopg2/setup.py'; execfile('/path/to/psycopg2/setup.py')" develop --no-deps failed with error code 1
Exception information:
Traceback (most recent call last):
  File "/path/to/acme/lib/python2.7/site-packages/pip-0.8.1-py2.7.egg/pip/basecommand.py", line 130, in main
    self.run(options, args)
  File "/path/to/acme/lib/python2.7/site-packages/pip-0.8.1-py2.7.egg/pip/commands/install.py", line 228, in run
    requirement_set.install(install_options, global_options)
  File "/path/to/acme/lib/python2.7/site-packages/pip-0.8.1-py2.7.egg/pip/req.py", line 1043, in install
    requirement.install(install_options, global_options)
  File "/path/to/acme/lib/python2.7/site-packages/pip-0.8.1-py2.7.egg/pip/req.py", line 534, in install
    self.install_editable(install_options, global_options)
  File "/path/to/acme/lib/python2.7/site-packages/pip-0.8.1-py2.7.egg/pip/req.py", line 617, in install_editable
    show_stdout=False)
  File "/path/to/acme/lib/python2.7/site-packages/pip-0.8.1-py2.7.egg/pip/__init__.py", line 249, in call_subprocess
    % (command_desc, proc.returncode))
InstallationError: Command /path/to/acme/bin/python -c "import setuptools; __file__='/path/to/psycopg2/setup.py'; execfile('/path/to/psycopg2/setup.py')" develop --no-deps failed with error code 1

Originally submitted by: Giles Brown

I'm working on something that uses copy_to/copy_expert to fetch query results into a file, but I'd like to be able to pass bind parameters so that I can feel comfortable about avoiding SQL injection problems in generating the query.

Originally submitted by: Matthew Ryan

This is extant in the current 2.4 release.

The current setup.py only allows setting the static-libpq option
via the setup.cfg, and doesn't accept it on the setup.py commandline.
Instead, you get the following error:
error: error in command line: command 'psycopg_build_ext' has no such option 'static_libpq'

I've attached a minimal patch which makes it work, tested on RHEL-5 and OSX.

Originally submitted by: mirco

Originally submitted as number 20 - http://psycopg.lighthouseapp.com/projects/62710/tickets/20

Install psycopg2 from binary package psycopg2-2.2.2.win32-py2.7-pg9.0.1-release.exe (Windows download from official site http://initd.org/psycopg/)
In some computer the psycopg2 works well with django test server but with Apache Server httpd fails to load some dll, perhaps because compiled against msvcrt.dll instead msvc90.dll with entries in error.log (for example “TemplateSyntaxError: Caught ImproperlyConfigured while rendering: 'django.db.backends.postgresql_psycopg2' isn't an available database backend.”)
The solution that I’ve found is to re-build psycopg2. These are the steps:
• Download and install MinGW. (Check for gcc.exe file in C:\MinGW\bin)
• Add C:\MinGW\bin to your PATH. (Right click My Computer | Advanced tab | Environment Variables | Select ‘Path’ from the ‘System variables’ list | Edit | Append ‘;C:\MinGW\bin’ to the end of the string in ‘Variable value’). Probably you need to reboot your computer.
• Download and execute the Postgre. SQL one-click installer.
• Download and extract psycopg2 (tar.gz archive with source codes)
• From within the extracted psycopg2 directory, execute the following command:
python setup.py build_ext --compiler=mingw32 build (Note: In command output check if there is msvcr90.dll instead of msvcrt.dll)
Your goodies will be in the local build directory (build\lib.win32-2.7\psycopg2). You can rename the installed psycopg2 directory C:\Python27\Lib\site-packages\psycopg2 with the local build one.
Someone suggests python setup.py build_ext –compiler=mingw32 install but I think it works if you haven’t already installed psycopg2.

Originally submitted by: fpoirotte

Hi,

To the best of my knowledge, there is currently no way in psycopg2 to make it use a specific pgpass file instead of the default one, when using keywords with the connect method.

What I'd like to be able to do is:
conn = psycopg2.connect(database="test", user="postgres", pgpass="/etc/testapp/.pgpass")

PostgreSQL accepts that kind of usage (see src/interfaces/libpq/fe-connect.c) since a very long time it seems and so I think it's only natural that psycopg2 accepts it too.

Also, I haven't tried it yet, but I think passing the path to the pgpass file in a conninfo string currently works (ie. psycopg2.connect("dbname=test user=postgres pgpass=/etc/testapp/.pgpass")). However, in my case this is not enough because I use some ORM wrapper which passes those information to psycopg2 as separate keyword arguments (and not as a single conninfo string).

Regards,
François

Originally submitted by: easiconseil

Originally submitted as number 6 - http://psycopg.lighthouseapp.com/projects/62710/tickets/6

i use rel-2.2.1 on Mac OS X 10.0.6 (SL) with Python 2.5
i want to use OpenERP 5 with PostGreSQL 8.4

when i build psycopg2, i have the message no appropriate architecture
when i build psycopg2 with FLAG -arch 86x64, i have this message
04/11/10 13:00:13 com.openerp.server[3667] ImportError: dlopen(/Library/Python/2.5/site-packages/openerp-server/psycopg2/_psycopg.so, 2): Symbol not found: _PQbackendPID

i dont understand.
i dont know where to lookup log files.
please, help me.

sorry for my english, i'm french ;-)
TIA

Originally submitted by: Marti Raudsepp

Originally submitted as number 24 - http://psycopg.lighthouseapp.com/projects/62710/tickets/24

psycopg 2.3.0 and 2.3.1 segfault immediately upon connect to the pgbouncer pooling middleware, because (apparently) pgbouncer does not pass on the DateStyle to the client.

The attached patch fixes the issue.

Originally submitted by: marko kreen

Originally submitted as number 9 - http://psycopg.lighthouseapp.com/projects/62710/tickets/9

when one statement issues several notices,
they appear in reversed order in .notices list. the reason is that they are prepended to temporary singly linked list which is later walked sequentially.

fix would be to append them to the singly linked list.

Originally submitted as number 2 - http://psycopg.lighthouseapp.com/projects/62710/tickets/2

Allow adaptation of Python dictionaries into hstore types and back.

Under development in piro's adapt-dict branch

Originally submitted by: ellonweb

I get this error when my project tries to import psycopg.
ImportError: DLL load failed: The specified module could not be found.

It traces back to this:
File "C:\Python26\lib\site-packages\psycopg2*init*.py", line 69, in
from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID

It's worth noting that this works fine from a python prompt, this error only occurs when used with Apache/mod_wsgi. It also works fine under Django's test server.

The files are all there, it's all on my Python path (otherwise I'd get a different type of import error). All the permissions are fine (I can install an older version of psycopg in place and it works fine).

I realize that this isn't necessarily a bug in psycopg but I've been unable to resolve the problem from speaking to people in #python or #django on freenode, and I've tried everything I can find on google! Was hoping you might be able to shed some more light on the issue.

psycopg 2.0.14 works fine. 2.2.1 and 2.3.2 both have this problem.

Running Windows, Python 2.6.5, Postgre 8.4.4, mod_wsgi 3.3, Django 1.2.3, Apache 2.2.17. I have a friend running my code with a similar setup and they have the same error.

Your help would be much appreciated.

From Eric Snow:

>>> cur.execute(";")
Traceback (most recent call last):
 File "<stdin>", line 1, in ?
SystemError: null argument to internal routine

Originally submitted by: marko kreen

Originally submitted as number 10 - http://psycopg.lighthouseapp.com/projects/62710/tickets/10

instead of passing column array to namedtuple() as array, it is comverted to string, which must be parsed immediately.

Originally submitted as number 11 - http://psycopg.lighthouseapp.com/projects/62710/tickets/11

The sdist doesn't include Makefile and runtests.py.

The two are probably too broken to be useful anyway, so eventually just document how to set up a testing environment.

Originally submitted by: Adam Petrovic

Hi,

I'm getting InterfaceError when selecting BYTEA data from custom function. Whole error is:
InterfaceError: can't receive bytea data from server >= 9.0 with the current libpq client library: please update the libpq to at least 9.0 or set bytea_output to 'escape' in the server config or with a query

All rows from "SELECT decode_fn(col).bytea_col" return empty "\x" data, but I can't reproduce it on simple "SELECT bytea_col".
libpq-9 installed a psycopg2 correctly linked.

Error is thrown in psycopg/typecast_binary.c line 170. I'm not C programmer, but is that IF right for empty data?

Originally submitted as number 4 - http://psycopg.lighthouseapp.com/projects/62710/tickets/4

Named tuples are cool!

In current Py3 implementation the typecaster receive a decoded string. This means that is basically impossible to have a binary typecaster in python.

Idea: add a "binary" flag (or a "type", currently only "text" and "binary" would be meaningful) to the typecaster type. It would default to text on Python 3 and either default to binary in Python 2 or not available at all. The create_type would grow an optional parameter with the proper default. Use the value to decide whether to pass a binary or a decoded text to the typecaster function.

Originally submitted by: Jon

Basically, a change in 2.4beta is an "optimization" such that a fetch of one row from a server-side (named) cursor results in the fetch of (up to) 2,000 rows. IMO, it's inappropriate for the underlying library to be making decisions like this - if I say I want one row I really only want one row to be fetched. PostgreSQL even includes the "cursor_tuple_fraction" config option, which defaults to 0.1, indicating that the expected nubmer of rows to be fetched from a server-side cursor is about 10% - and the optimizer makes decisions based upon that information. Applications themselves can start using the fetch-more-than-one-row API if that's what they want (and what they should be doing) -- SQLAlchemy, for example, does just that. Making the decision to fetch /more/ rows than asked is not something that should be done at this level, IMO -- I asked for a server-side cursor for a reason.

Thanks for hearing me out.

Originally submitted by: Andrei Antoukh

Since I upgraded from 2.3 to 2.4 psycopg'm having a problem with my django web applications, connect to the database.

When you synchronize (Django), I get this error: http://pastebin.com/hQ2s5nyQ

Note: This error happens in my linux systems with postgresql 9.x, but freebsd8.2 with the same version of postgresql works correctly.
The python version is 2.7 in both cases.

Currenty _pq_copy_out_v3 is using PyObject_CallFunction(func, "s#", buffer, len) to pass data read from the db to the file: this implies utf8 encoding, so would break on different client encodings.

Shall we pass unicode or bytes? Shall we check if the file has an encoding?

Originally submitted by: Jean-Baptiste Quenot

Originally submitted as number 15 - http://psycopg.lighthouseapp.com/projects/62710/tickets/15

Patch at http://dl.dropbox.com/u/5836407/Patches/Psycopg2/0005-Mention-connection-object-in-debug-logs.patch

Helps to identify the underlying connection for the active thread in the debug logs.

Originally submitted by: Jon

Originally submitted as number 23 - http://psycopg.lighthouseapp.com/projects/62710/tickets/23

This is what I get when I try to compile on CentOS 5.5 x86_64:

gcc -pthread -shared build/temp.linux-x86_64-2.4/psycopg/psycopgmodule.o build/temp.linux-x86_64-2.4/psycopg/pqpath.o build/temp.linux-x86_64-2.4/psycopg/typecast.o build/temp.linux-x86_64-2.4/psycopg/microprotocols.o build/temp.linux-x86_64-2.4/psycopg/microprotocols_proto.o build/temp.linux-x86_64-2.4/psycopg/connection_type.o build/temp.linux-x86_64-2.4/psycopg/connection_int.o build/temp.linux-x86_64-2.4/psycopg/cursor_type.o build/temp.linux-x86_64-2.4/psycopg/cursor_int.o build/temp.linux-x86_64-2.4/psycopg/lobject_type.o build/temp.linux-x86_64-2.4/psycopg/lobject_int.o build/temp.linux-x86_64-2.4/psycopg/notify_type.o build/temp.linux-x86_64-2.4/psycopg/xid_type.o build/temp.linux-x86_64-2.4/psycopg/adapter_qstring.o build/temp.linux-x86_64-2.4/psycopg/adapter_pboolean.o build/temp.linux-x86_64-2.4/psycopg/adapter_binary.o build/temp.linux-x86_64-2.4/psycopg/adapter_asis.o build/temp.linux-x86_64-2.4/psycopg/adapter_list.o build/temp.linux-x86_64-2.4/psycopg/adapter_datetime.o build/temp.linux-x86_64-2.4/psycopg/adapter_pfloat.o build/temp.linux-x86_64-2.4/psycopg/adapter_pdecimal.o build/temp.linux-x86_64-2.4/psycopg/green.o build/temp.linux-x86_64-2.4/psycopg/utils.o -L/usr/lib64 -lpq -o build/lib.2.4/psycopg2/_psycopg.so
/usr/bin/ld: build/temp.linux-x86_64-2.4/psycopg/psycopgmodule.o: relocation R_X86_64_PC32 against `psyco_typecast_datetime_init' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value

Originally submitted by: Paul Reznicek

Originally submitted as number 19 - http://psycopg.lighthouseapp.com/projects/62710/tickets/19

Hello,
There are no more isready and fileno in cursor, so that using select to wait for notify make problems. Fileno can be used from connection, but without isready the waiting for notification is difficult.

Originally submitted by: Marco Cassiano

Originally submitted as number 8 - http://psycopg.lighthouseapp.com/projects/62710/tickets/8

N.B Errore già presente anche sulla 2.2.2

select.select (indipendentemente dalla versione di Postgres, 8.4 o 9.0.1) fallisce sulla referenziazione del cursore, peraltro aperto con successo.
Con la versione 2.2.1 funziona correttamente.

Codice e segnalazione :

import mdnlib
import psycopg2,psycopg2.extensions
import select # per la gestione del notify

def test_notify():
print 'Notify'
c = psycopg2.connect(host = d['host'], database=d['dbname'], user=d['user'], password=psw,port=d['port'])
c.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
cursore = c.cursor()

cur0 = c.cursor()
cur0.execute('LISTEN "scheduler"')
cur0.execute('LISTEN "schedulerKill"')
print('In attesa di notify "scheduler" e "schedulerKill";')
TIMEOUT = 8
i=1
while(i):
    i +=1 

l’errore avviene sulla prossima riga

    if select.select([cur0],[],[], TIMEOUT)==([],[],[]):
        #self.logger.debug("Timeout(''keepalive'')!")
        pass
    else:
        if cur0.isready():
            lnotify = cur0.connection.notifies.pop()
            print ("Got NOTIFY: %s " % str(lnotify))
            if lnotify[1] == "schedulerKill": print 'self.kill()'
            elif lnotify[1] == "scheduler":   print 'self.run()'
            else: print ("Orrore: notify inaspettato !!!!")
    if i == 2: i=0            

if name == 'main':
d = mdnlib.dns_postgres_nativo.copy()
test_notify()

______________________________________________ errore:___________________
Notify
In attesa di notify "scheduler" e "schedulerKill";
Traceback (most recent call last):
File "\intranetsvil\discoe tmp\bugPsycopg.py", line 39, in
test_notify()
File "\intranetsvil\discoe tmp\bugPsycopg.py", line 22, in test_notify
if select.select([cur0],[],[], TIMEOUT)==([],[],[]):
TypeError: argument must be an int, or have a fileno() method.

Process completed, Exit Code 1.
Execution time: 00:03.151

Originally submitted by: Pumukel

Hello,

The ZPsycopgDA is broken, and can't be used in Zope 2.12.9+ the reference for Shared/DC/ZRDB/www/DBAdapter_icon.gif did not longer exist.

following nabble Report shows a solution:
http://old.nabble.com/Zope-2.12.9-broke-ZMySQLDA-in-the-middle-of-a-stable-release-series-td29151717.html

Originally submitted by: Jean-Baptiste Quenot

Originally submitted as number 12 - http://psycopg.lighthouseapp.com/projects/62710/tickets/12

Patch at http://dl.dropbox.com/u/5836407/Patches/Psycopg2/0001-Add-isolation-level-for-BEGIN-READ-ONLY.patch

See http://www.postgresql.org/docs/9.0/static/sql-set-transaction.html. A read-only SQL transaction cannot alter non-temporary tables.

Can be for the version we compiled against and the runtime one since pg 9.1.

Originally submitted by: marplatense

Using psycopg2 2.4 beta2 with python 2.6: when I insert an empty array (tested with timestamp array) it seems it's casted to text by psycopg2 and, consequently, raises a ProgrammingError because of wrong type. This did not happen with psycopg2 version 2.3.2. I attach a full example with this report, demonstrating the behavior with psycopg2 2.3 and 2.4

Originally submitted by: Anders Pearson

Briefly, the issue is that when installing psycopg2 in a virtualenv that was created with --no-site-packages (in an attempt to be completely independent of the underlying system), psycopg2's setup.py will still find mx and set HAVE_MXDATETIME=1. The compilation and install succeed, but since mx isn't installed in the virtualenv, importing psycopg2 results in an ImportError: can't import mx.DateTime module.

As far as I can tell, this happens because setup.py does

mxincludedir = os.path.join(get_python_inc(plat_specific=1), "mx")

to determine whether mx is available. That will find the mx include directory (eg, /usr/include/python2.6/mx) and it will be able to compile. But this is circumventing the virtualenv and isn't safe behavior when installing into a virtualenv.

I can easily set mxincludedir to a bogus directory to trick it into thinking mx isn't installed anywhere and get it to work, but that doesn't strike me as the best and cleanest solution.

I'm not sure exactly what the best solution is overall, but I'd suggest that one way or another, psycopg2 should only compile in mx.DateTime support when it can both find the mx headers and know that the mx python library will be available.

The segfault is still in the python2/python3 repos after merged the 'noleak' branch.

Segfault with python 2.5 64 bits compiled with gcc 4.3.2, but not with 4.4.5.

Originally submitted as number 5 - http://psycopg.lighthouseapp.com/projects/62710/tickets/5

An object can't be adapted even if one of its superclasses can: this breaks the Liskov invariance as the subclass is expected to behave like the superclass. In practice, for instance, if s.b. defines a string subclass, it is likely he wants it to be serialized in the database as a string. Currently psycopg raises a "can't adapt" (at least now it tells the class, so debugging is easier).

Proposal: if an object b of type B can't be adapted (it would be an error right now), walk its type's MRO and, if an adapter f for a supertype A is found, register the same adapter f for B too. If possible, detect the scope in which the adapter is registered for A (global, connection, cursor) and register the f as adapter for B in the same scope.

Originally submitted by: Landreville

When running python setup.py build_ext on FreeBSD I get the following error:
gcc -pthread -shared ${LDFLAGS} build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/psycopgmodule.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/green.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/pqpath.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/utils.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/bytes_format.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/connection_int.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/connection_type.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/cursor_int.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/cursor_type.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/lobject_int.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/lobject_type.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/notify_type.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/xid_type.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_asis.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_binary.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_datetime.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_list.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_pboolean.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_pdecimal.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_pfloat.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/adapter_qstring.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/microprotocols.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/microprotocols_proto.o build/temp.freebsd-7.0-RELEASE-i386-2.7/psycopg/typecast.o -L/usr/local/pgsql-9.0.3/lib -lpq -o build/lib.freebsd-7.0-RELEASE-i386-2.7/psycopg2/_psycopg.so
gcc: ${LDFLAGS}: No such file or directory
error: command 'gcc' failed with exit status 1

This is with psycopg2-2.4-beta2 (same thing happens with 2.3.2), Postgres 9.0.3, Python 2.7.1, FreeBSD 7

microprotocols_adapt has a fast path returning NULL on None. This is wrong: it should return an object conform to ISQLQuote. It goes unnoticed because _mogrify has a fast path not calling microprotocol_getquoted, but the bug emerges trying to adapt a record containing NULL.

In [6]: cur.mogrify("""select %s;""", [ tuple((1,'hi',None)) ])---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)

/home/piro/dev/psycopg2/<ipython console> in <module>()

/home/piro/dev/psycopg2/build/lib.2.6/psycopg2/extensions.pyc in getquoted(self)
    123             if hasattr(obj, 'prepare'):
    124                 obj.prepare(self._conn)
--> 125         qobjs = [str(o.getquoted()) for o in pobjs]
    126         return '(' + ', '.join(qobjs) + ')'
    127 

It can be fixed by:

1. dropping the special case in microprotocol_adapt and adding an adapter such as:

# untested
extensions.register_adapter(type(None), lambda x: exteions.AsIs("NULL"))

it would still be a fast path in mogrify, but would allow uniform treatment of None in composite types;

1. (I mean 2 but markdown sucks) Put a different fastpath in adapt: always return the same AsIs("NULL").
2. (three) adding a special case to SQL_IN. No, I don't like it. This is the solution currently adopted by adapter_list.

Note: grep -r '"NULL"' psycopg reveals that AsIs adapts None to "NULL". This is either unnecessary or we can use that... in either case we are a little bit schizophrenic.

Originally submitted by: Nate C

Originally submitted as number 7 - http://psycopg.lighthouseapp.com/projects/62710/tickets/7

after this ...

import psycopg2
from decimal import Decimal

conn = psycopg2.connect('dbname=test')
cursor = conn.cursor()
cursor.mogrify('select %s', (Decimal('3.27'),))

.. python crashes

Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32

pyscopg2.version
2.2.2 (dt dec ext pq3)

Originally submitted by: piro

Originally submitted as number 17 - http://psycopg.lighthouseapp.com/projects/62710/tickets/17

Test ticket to understand lighthouse subscriptions: please ignore.

Originally submitted by: Jean-Baptiste Quenot

Originally submitted as number 13 - http://psycopg.lighthouseapp.com/projects/62710/tickets/13

Patch at http://dl.dropbox.com/u/5836407/Patches/Psycopg2/0002-Add-socket_fd-to-connection-object.patch

Originally submitted by: IZ

You use the following query to detect the presence of the hstore type in a database:

SELECT t.oid, %s
FROM pg_type t JOIN pg_namespace ns
ON typnamespace = ns.oid
WHERE typname = 'hstore' and nspname = 'public';

(lines 684-688 of psycopg2/extras.py, function get_oids(self, conn_or_curs))

which is not entirely correct as hstore may be installed in difference schema (but search_path modified properly so that every client sees it), which is actually a good practice (experienced PostgreSQL admins install contribs in their own schemas not to make a big mess in public), though not default behaviour (hstore.sql by default comes to public schema, yes). So I would encourage you to remove "and nspname = 'public'" of the query above, unless you have strong reasons to hold it there.

Many thanks to you for your great work.

Originally submitted as number 16 - http://psycopg.lighthouseapp.com/projects/62710/tickets/16

We can drop two roundtrip with the server per connection:

• SET DATESTYLE TO 'ISO' is retundant if we detect that the DateStyle setting is as we like, and we can query it with PQparameterStatus without a roundtrip.
• client_encoding can be read from PQparameterStatus as well, no need for SHOW.

Originally submitted by: Marti Raudsepp

Currently Pyscopg does not properly convert Python float negative infinity to PostgreSQL:

  >>> c.mogrify('SELECT %s', [float('-inf')])
  "SELECT 'Infinity'::float"

This patch fixes the issue and converts it into '-Infinity'. I really hope this makes it into the next release.