rogerlew / pyvttbl Goto Github PK

Document sphericity calculations, generalized eta, power, loftus and masson 
error bars.

Describe the feature you would like to see:

Incorporate ensemble plotting routine

What steps will reproduce the problem?
1. pip install pyvttbl
2. from pyvttbl import DataFrame

What is the expected output? What do you see instead?
There shouldn't be any error


What version of the product are you using? On what operating system?
0.3.6.1 with Python 2.7


Please provide any additional information below.
Seems like qsturng is available from http://code.google.com/p/qsturng-py/ but 
it is not available as a package. Is this something that should have been 
included in the pyvttbl package?

mportError                               Traceback (most recent call last)
/Users/hdara/src/ma/<ipython-input-1-83de34da1e0f> in <module>()
----> 1 from pyvttbl import DataFrame

/Users/hdara/bin/python2.7/lib/python2.7/site-packages/pyvttbl/__init__.py in 
<module>()
      2 # This software is funded in part by NIH Grant P20 RR016454.

      3 
----> 4 from pyvttbl import __version__, DataFrame, PyvtTbl, Ttest, Anova1way, \
      5      Correlation, ChiSquare1way, ChiSquare2way, Marginals, Descriptives, \
      6      Histogram

/Users/hdara/bin/python2.7/lib/python2.7/site-packages/pyvttbl/pyvttbl.py in 
<module>()
     29 from texttable import Texttable as TextTable
     30 from stats import jsci, stats, pstat
---> 31 from qsturng import qsturng, psturng
     32 
     33 from anova import Anova

ImportError: No module named qsturng

What steps will reproduce the problem?
1. make sparse pivot table (e.g. some elements empty)
2. do .to_dataframe() on sparse pivot table
3. crash with message:

Traceback (most recent call last):
  File "pilot-cancelation-report.py", line 148, in <module>
    a.run()
  File "pilot-cancelation-report.py", line 142, in run
    d1 = df.pivot('ks', rows=['duvod_popis'], cols=['zruseno_rok','zruseno_tyden'], aggregate='sum').to_dataframe()
  File "/usr/lib/python2.7/site-packages/pyvttbl-0.5.2.2-py2.7.egg/pyvttbl/base.py", line 2374, in to_dataframe
    df.insert(zip(header, rdata))
  File "/usr/lib/python2.7/site-packages/pyvttbl-0.5.2.2-py2.7.egg/pyvttbl/base.py", line 1554, in insert
    self[k]=np.concatenate((self[k], np.array([v], dtype=self._get_nptype(k))))
TypeError: long() argument must be a string or a number, not 'NoneType'

I have dirty-fixed this with (around line 1553 of base.py):

        elif c - s == set():
            for (k, v) in OrderedDict(row).items():
                if v is None:
                    v = 0
                self[k]=np.concatenate((self[k], np.array([v], dtype=self._get_nptype(k))))
                self.conditions[k].add(v)

(if v is None, let v = 0). But this is only dirty fix.

What is the expected output? What do you see instead?

it should generate DataFrame object.

What version of the product are you using? On what operating system?

0.5.2.2
linux (debian)
python 2.7

Please provide any additional information below.

more info on request, I'm able provide example data, if necessary.

Describe the feature you would like to see:

document tool for restructuring data form the long to the wide format.

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyvttbl/stats/_anova.pyc in _num2binvec(self, d, p)
   1238                 d=floor(d/2.)
   1239
-> 1240         return list(array(list(zeros((p-len(b))))+b)+1.)
   1241
   1242 ##    def output2html(self, fname, script=''):

TypeError: 'float' object cannot be interpreted as an index

What steps will reproduce the problem?

Incorporate multi-subplot timeseries plotting

What steps will reproduce the problem?
1. Create a CSV with a column only containining 0.00 value for each row
2. read_tbl in DataFrame
3. Query types

What is the expected output? What do you see instead?
I expected to get real to attach this table with another where this column type 
is real

Python 2.7.6 (default, Dec 12 2013, 14:16:02) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
pyvttbl 0.5.2.2

Describe the feature you would like to see:

Incorporate personal collection of timeseries routines into pyvttbl

Describe the feature you would like to see:
Link the datafiles in the examples to the documentation.

What steps will reproduce the problem?

Trying to use a unicode string that contains a non-ascii character, as a value 
on a record.

What is the expected output?

Successful instantiation of a DataFrame object

What do you see instead?

An exception caused by trying to convert a non-ascii unicode string into ascii:

Traceback (most recent call last):
  File "non_ascii_char_table_header.py", line 19, in <module>
    for col_name, col in zip(header, zip(*records))
  File "/home/jerojasro/.virtualenvs/zinobe_dashboard/local/lib/python2.7/site-packages/pyvttbl/base.py", line 86, in __init__
    super(DataFrame, self).update(*args, **kwds)
  File "/home/jerojasro/.virtualenvs/zinobe_dashboard/lib/python2.7/_abcoll.py", line 500, in update
    self[key] = value
  File "/home/jerojasro/.virtualenvs/zinobe_dashboard/local/lib/python2.7/site-packages/pyvttbl/base.py", line 347, in __setitem__
    __setitem__(key, np.array(item, dtype=self._get_nptype(key)))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 1: 
ordinal not in range(128)


What version of the product are you using? On what operating system?

pyvttbl==0.5.2.2 , installed through pip, on Ubuntu Linux


Please provide any additional information below.


See attached files:

non_ascii_char_in_record.py : example code that causes the crash reported

string_encoding_patch : suggested patch to fix the issue

What steps will reproduce the problem?
1. Create a simple CSV file (comma-separated) with a space in one of its column 
names
2. Try to read it with read_tbl()
3. Observer!

What is the expected output? What do you see instead?

The file should be read correctly. Instead:
>>> df.read_tbl('test.csv')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build\bdist.win32\egg\pyvttbl\pyvttbl.py", line 247, in read_tbl
  File "build\bdist.win32\egg\pyvttbl\pyvttbl.py", line 324, in __getitem__
KeyError: 'a'


What version of the product are you using? On what operating system?
0.3.6.1 on Windows.

1. Put bz2 python 2.7 in /opt/downloads
2. bunzip2 *bz2
3. cd Py*
4. ./configure --prefix=/opt/plonepython27
5. make install
6. download setuptools in the bin of python: wget 
http://peak.telecommunity.com/dist/ez_setup.py
7. ./python2.7 ez_setup.py
8. Now we can install with easy_install: ./easy_install pyvttbl 
9. apt-get install libxtst-dev (more might be needed - this is just my ubuntu 
system)
9. ./easy_install numpy
10. ./easy_install scipy
11. ./easy_install matplotlib    (this is the pylab error fix)
12. Now you can use csv files you can download together with the code: hg clone 
https://code.google.com/p/pyvttbl/ 
13. And do examples from here: http://code.google.com/p/pyvttbl/

Potentially the rnames attribute of a pyvttbl can be longer than the data of 
the pyvttbl, when a multi-dimensional pivot is performed on a dataframe.  The 
resulting pyvttbl will not write out or print and the row labels may be 
mismatched with the data of the pyvttbl.

Please note this error was discovered on a much more complex data set, but the 
example provided is a simple case for illustration purposes.

What steps will reproduce the problem?
1. load a table into a dataframe in which there is not data for every 
permutation of two fields - example:

>>> df = DataFrame()
>>> df.insert({'id':0,'Name':'name1','Year':2010,'member':'Y'})
>>> df.insert({'id':1,'Name':'name1','Year':2011,'member':'N'})
>>> df.insert({'id':2,'Name':'name2','Year':2011,'member':'Y'})
>>> print df

member   Name    id   Year 
==========================
Y        name1    0   2010 
N        name1    1   2011 
Y        name2    2   2011

2. pivot the dataframe using two row labels or column labels - example:

>>> my_pivot = df.pivot('id',rows = ['Name','Year'], cols = ['member'], 
aggregate='count')

What is the expected output? What do you see instead?

this resulting pyvt table will not print or write because it has more row names 
than date rows - example 

>>>print my_pivot

Traceback (most recent call last):
  File "<pyshell#76>", line 1, in <module>
    print my_pivot
  File "C:\Python27\lib\site-packages\pyvttbl-0.3.6.7-py2.7.egg\pyvttbl\pyvttbl.py", line 2355, in __str__
    self[i] +
IndexError: list index out of range

>>> len(my_pivot.rnames)
4
>>> len(my_pivot)
3

>>> for name in my_pivot.rnames:
    print name


[('Name', u'name1'), ('Year', 2010)]
[('Name', u'name1'), ('Year', 2011)]
[('Name', u'name2'), ('Year', 2010)]
[('Name', u'name2'), ('Year', 2011)]

>>> for row in my_pivot:
    print row


[0, 1]
[1, 0]
[0, 1]


In the above example, the row name  - [('Name', u'name2'), ('Year', 2010)] - is 
not necessary or reflective of the original data.

What version of the product are you using? On what operating system?
pyvttbl version 0.3.6.7
Windows XP
Python 2.7.2

Please provide any additional information below.

Thanks for your help!

We really like this python module!

I was wondering if you could help me with the following:

After making a DF,...

   df = pyvttbl_DataFrame()
   df.read_tbl('/home/brain/tmp_csv.csv')

...And then fitting an ANOVA model...

   aov = df.anova('dat', bfactors='AgeGroup',
                         wfactors=['withinORcross','modality'])

...I get a 'TypeError: can only concatenate list (not "str") to list 
print(aov)' error (copied out in full below). 

I believe this may be something to do with the organization and formatting of 
the .csv file, since the ANOVA tutorial example works fine. In this case my 
.csv file was written to file from pandas; not quite sure how that could be the 
problem per se though. 

Thanks very much, 

John





---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-147-1f30da5cce82> in <module>()
      2 df = pyvttbl_DataFrame()
      3 df.read_tbl('/home/brain/tmp_csv.csv')
----> 4 aoc = df.anova('dat', bfactors='AgeGroup', wfactors=['withinORcross', 
'modality'])

/usr/local/lib/python2.7/dist-packages/pyvttbl-0.5.2.2-py2.7.egg/pyvttbl/base.py
c in anova(self, dv, sub, wfactors, bfactors, measure, transform, alpha)
   1973         aov=stats.Anova()
   1974         aov.run(self, dv, sub=sub, wfactors=wfactors, bfactors=bfactors,
-> 1975                 measure=measure, transform=transform, alpha=alpha)
   1976         return aov
   1977 

/usr/local/lib/python2.7/dist-packages/pyvttbl-0.5.2.2-py2.7.egg/pyvttbl/stats/_
anova.pyc in run(self, dataframe, dv, wfactors, bfactors, sub, measure, 
transform, alpha)
    643 
    644         # a list of all the factors
--> 645         factors=wfactors+bfactors
    646         self.dftrim=0.
    647 

TypeError: can only concatenate list (not "str") to list

http://pyvttbl.googlecode.com/hg/doc/_build/html/Anova.html#example-mixed-subjec
ts-anova
-- 


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-147-1f30da5cce82> in <module>()
      2 df = pyvttbl_DataFrame()
      3 df.read_tbl('/home/brain/tmp_csv.csv')
----> 4 aoc = df.anova('dat', bfactors='AgeGroup', wfactors=['withinORcross', 
'modality'])

/usr/local/lib/python2.7/dist-packages/pyvttbl-0.5.2.2-py2.7.egg/pyvttbl/base.py
c in anova(self, dv, sub, wfactors, bfactors, measure, transform, alpha)
   1973         aov=stats.Anova()
   1974         aov.run(self, dv, sub=sub, wfactors=wfactors, bfactors=bfactors,
-> 1975                 measure=measure, transform=transform, alpha=alpha)
   1976         return aov
   1977 

/usr/local/lib/python2.7/dist-packages/pyvttbl-0.5.2.2-py2.7.egg/pyvttbl/stats/_
anova.pyc in run(self, dataframe, dv, wfactors, bfactors, sub, measure, 
transform, alpha)
    643 
    644         # a list of all the factors
--> 645         factors=wfactors+bfactors
    646         self.dftrim=0.
    647 

TypeError: can only concatenate list (not "str") to list