I'm used to Python being easier than this

I've been using python for several years now so I've pretty much got my Python environment set up so that it's easy and fast to use. Recently though I've been doing some data wrangling and have been following a few tutorials which rely on Anaconda or Miniconda. In order to minimise the potential for my environment causing issues (and because I wanted to have a play with them) I installed first Anaconda and subsequently Miniconda.

Installing and uninstalling conda

One of Condas selling points is that it is very easy both to install and uninstall, go to the website download the appropriate file and follow the instructions. Want rid of it? Delete the folder (usually located in your home directory), remove the lines it's added to your .bash_profile and it's gone. The first thing I noticed after installing Conda was that it messed with my Virtualenv setup, this wasn't a big deal just a touch confusing as every time I launched a Virtulaenv it announced:

Error: deactivate must be sourced. Run 'source deactivate'
instead of 'deactivate'.

Since this wasn't actually affecting my ability to deactivate Virtualenv's by typing deactivate I lived with it. Where I hit a problem was when I tried to install Pandas. This should be easy after all it's both one of the things Conda is intended for and the recommended way to install Pandas. I've used the command line for so long that I totally failed to get my head around Anaconda which seems to be a lot more visual than I'm used to, so I uninstalled it and installed Miniconda instead. This was better as I understood what I was doing here.

It failed.

Horribly. Here's what I did so you can give it a try if you wish:

1. Download the appropriate Conda install file (I used 3.6): https://conda.io/miniconda.html.

2. Follow the installation instructions: https://conda.io/docs/user-guide/install/macos.html. For Miniconda this means running bash Miniconda3-latest-MacOSX-x86_64.sh.

3. run source .bash_profile (or restart your terminal window).

4. Test the installation: https://conda.io/docs/user-guide/install/test-installation.html by running conda list.

   

   It works so...

5. back to the instructions on the Pandas website.

6. Create a virtual environment: conda create -n name_of_my_env python

7. activate it: activate name_of_my_env

8. run conda install pandas

9. test your install:

To do this you'll need to conda install pytest then launch python and:

>>> import pandas as pd
>>> pd.test()

Here's what I got:

running: pytest --skip-slow --skip-network /Users/NotQuiteHere/miniconda3/envs/data_vis/lib/python3.6/site-packages/pandas
================================== test session starts ===================================
platform darwin -- Python 3.6.4, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /Users/NotQuiteHere, inifile:
collected 0 items / 1 errors

========================================= ERRORS =========================================
___________________________________ ERROR collecting  ____________________________________
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/config.py:328: in _getconftestmodules
    return self._path2confmods[path]
E   KeyError: local('/Users/NotQuiteHere/miniconda3/envs/data_vis/lib/python3.6/site-packages/pandas/tests/io')

During handling of the above exception, another exception occurred:
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/config.py:359: in _importconftest
    return self._conftestpath2mod[conftestpath]
E   KeyError: local('/Users/NotQuiteHere/miniconda3/envs/data_vis/lib/python3.6/site-packages/pandas/tests/io/conftest.py')

During handling of the above exception, another exception occurred:
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/config.py:365: in _importconftest
    mod = conftestpath.pyimport()
../miniconda3/envs/data_vis/lib/python3.6/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:213: in load_module
    py.builtin.exec_(co, mod.__dict__)
../miniconda3/envs/data_vis/lib/python3.6/site-packages/pandas/tests/io/conftest.py:3: in <module>
    import moto
E   ModuleNotFoundError: No module named 'moto'

During handling of the above exception, another exception occurred:
../miniconda3/envs/data_vis/lib/python3.6/site-packages/py/_path/common.py:377: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../miniconda3/envs/data_vis/lib/python3.6/site-packages/py/_path/common.py:429: in gen
    for p in self.gen(subdir):
../miniconda3/envs/data_vis/lib/python3.6/site-packages/py/_path/common.py:418: in gen
    dirs = self.optsort([p for p in entries
../miniconda3/envs/data_vis/lib/python3.6/site-packages/py/_path/common.py:419: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/main.py:765: in _recurse
    ihook = self.gethookproxy(path)
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/main.py:669: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/config.py:342: in _getconftestmodules
    mod = self._importconftest(conftestpath)
../miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/config.py:367: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: ModuleNotFoundError("No module named 'moto'",)
E     File "/Users/NotQuiteHere/miniconda3/envs/data_vis/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 213, in load_module
E       py.builtin.exec_(co, mod.__dict__)
E     File "/Users/NotQuiteHere/miniconda3/envs/data_vis/lib/python3.6/site-packages/pandas/tests/io/conftest.py", line 3, in <module>
E       import moto
!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!
================================ 1 error in 0.62 seconds =================================

Yeah that's not right... Ok lets try installing moto and see if that fixes it: conda install moto

    Fetching package metadata ...........

    PackageNotFoundError: Packages missing in current channels:

      - moto

    We have searched for the packages in the following channels:

      - https://repo.continuum.io/pkgs/main/osx-64
      - https://repo.continuum.io/pkgs/main/noarch
      - https://repo.continuum.io/pkgs/free/osx-64
      - https://repo.continuum.io/pkgs/free/noarch
      - https://repo.continuum.io/pkgs/r/osx-64
      - https://repo.continuum.io/pkgs/r/noarch
      - https://repo.continuum.io/pkgs/pro/osx-64
      - https://repo.continuum.io/pkgs/pro/noarch

So, no. Right well it must be available via pip: pip install moto

Right now we have moto, lets go back to testing Pandas: um...

I started reading the errors and quickly realised that I didn't actually have the time or energy to properly identify and attempt to fix whatever it was in Conda that was messing up. Nor did a cursory search of the internet give me any useful fixes.

Back to my usual tool chain.

1. Spin up a venv (I use virtualenvwrapper which I highly recommend, particularly as it works with tab complete) mkvirtualenv env_name. Virtualenvwrapper automatically starts it for you.
2. Install Pandas and pytest: pip install pandas pytest
3. Try testing it: nope, still needs moto: pip install moto
4. Try testing again, it works!

There's still errors (these particular errors are things that the test writer expected to fail) and warnings so it's not a perfect pass but I've now got a usable install \o/.