ENH/BUG: pd.read_csv(): usecols does not work when also using skipfooter

[89465127]

Steps to reproduce:

>>> import pandas as pd
>>> pd.__version__
'0.10.1'
>>> import StringIO
>>> data = 'a,b,c,d\n1,2,3,foo\n4,5,6,bar\n7,8,9,baz'
>>> df = pd.read_csv(StringIO.StringIO(data), usecols=[1])
>>> df
   b
0  2
1  5
2  8
>>> df = pd.read_csv(StringIO.StringIO(data), usecols=[1], skipfooter=1)
>>> df
   a  b  c    d
0  1  2  3  foo
1  4  5  6  bar

[jreback]

right now (even in master & 0.12), skip_footer forces the python parser and usecols is not implemented there. So best to simply read in all of your data (with usecols) and then subset it, e.g.

df.iloc[0:-1]

[guyrt]

@jreback I've been looking at this one. I'm close, but it's become a game of wack-a-mole. There are a lot of interactions between headers, name, and usecols. I'll post a bigger example of those interactions over the weekend.

[jreback]

np @guyrt thanks for the attention! you are really nailing down these csv buggies! lmk

[jreback]

@guyrt any luck with this?

[guyrt]

I've started working, but it's still very much a WIP.

https://github.com/guyrt/pandas/tree/issue-4355-usecols-python-engine

[jreback]

@guyrt how's this coming?

[guyrt]

Still WIP. I moved from Toronto to Seattle over the last two days, so I've been busy :) Should have time to clean it up this afternoon.

[jreback]

@guyrt big move! gr8 ...thanks