Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compilation error when installing python requirements, succeeds without explicit csvkit version #16268

Open
gabegm opened this issue Apr 20, 2021 · 0 comments
Open

Compilation error when installing python requirements, succeeds without explicit csvkit version #16268

gabegm opened this issue Apr 20, 2021 · 0 comments

Comments

@gabegm
Copy link

gabegm commented Apr 20, 2021

Context

OS: Ubuntu 16.04 x64

Steps to Reproduce

  1. Follow the steps from the official docs here untill the python_requirements.py step
  2. Run $ sudo pip install --no-use-wheel -r python_requirements.txt, dependency building should fail
  3. Install each dependency individually, i.e. $ sudo pip install LIBRARY==VERSION
  4. $ sudo pip install csvkit==0.9.0 should fail
  5. Remove the version from the csvkit requirement, i.e. csvkit
  6. Rerun $ sudo pip install --no-use-wheel -r python_requirements.txt, this should now build successfully

Current Result

python_requirements.txt failure

$ sudo pip install --no-use-wheel -r python_requirements.txt
DEPRECATION: --no-use-wheel is deprecated and will be removed in the future.  Please use --no-binary :all: instead.
The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): chardet==2.3.0 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): argparse==1.2.1 in /usr/lib/python2.7 (from -r python_requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): brewery==0.6 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 3))
Requirement already satisfied (use --upgrade to upgrade): redis==2.4.9 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 4))
Requirement already satisfied (use --upgrade to upgrade): python-varnish==0.1.2 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 5))
Collecting gdal==2.2.2 (from -r python_requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/11/cd/edef955a6ef6cb4f2113647d7465b34663e074451b66919e734466ba5197/GDAL-2.2.2.tar.gz (475kB)
    100% |████████████████████████████████| 481kB 2.5MB/s 
Collecting csvkit==0.9.0 (from -r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/37/39/e8a2bd3fe4c132fb0a09f7e8382341c4aeda2144506518465eccb4ffa655/csvkit-0.9.0.tar.gz
Collecting openpyxl==2.1.3 (from -r python_requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/86/7a/bba821c554b5fbde20aaa70cf5ae8c284883f38162deee7852e068db6246/openpyxl-2.1.3.tar.gz (96kB)
    100% |████████████████████████████████| 102kB 10.7MB/s 
Collecting xlrd>=0.7.1 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/a6/b3/19a2540d21dea5f908304375bd43f5ed7a4c28a370dc9122c565423e6b44/xlrd-2.0.1.tar.gz (100kB)
    100% |████████████████████████████████| 102kB 10.7MB/s 
Collecting sqlalchemy>=0.6.6 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/29/25/1fbcbf47928731ff46cbf3eaaaacbdcc042dd85f919580055f30a0dbdb0a/SQLAlchemy-1.4.9.tar.gz (7.5MB)
    11% |███▊                            | 880kB 45.4MB/s eta 0:00:01debug2: channel 0: window 998845 sent adjust 49731
    76% |████████████████████████▋       | 5.8MB 43.1MB/s eta 0:00:01debug2: channel 0: window 999409 sent adjust 49167
    100% |████████████████████████████████| 7.5MB 168kB/s 
  Running setup.py (path:/tmp/pip-build-56Rm7O/sqlalchemy/setup.py) egg_info for package sqlalchemy produced metadata for project name unknown. Fix your #egg=sqlalchemy fragments.
Collecting six>=1.6.1 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/6b/34/415834bfdafca3c5f451532e8a8d9ba89a21c9743a0c59fbd0205c7f9426/six-1.15.0.tar.gz
Collecting python-dateutil==2.2 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/75/c5/85d027471fa665f8c8b8eb0b925f9d84b4eee745a257b16de4957de99e81/python-dateutil-2.2.tar.gz (259kB)
    100% |████████████████████████████████| 266kB 4.7MB/s 
Collecting dbf==0.94.003 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/7c/ef/79873517838910e8f1bd5610ab3de380a3886675103072727266eec91bbc/dbf-0.94.003.tar.gz (79kB)
    100% |████████████████████████████████| 81kB 10.5MB/s 
Collecting jdcal (from openpyxl==2.1.3->-r python_requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/7b/b0/fa20fce23e9c3b55b640e629cb5edf32a85e6af3cf7af599940eb0c753fe/jdcal-1.4.1.tar.gz
Installing collected packages: gdal, xlrd, unknown, jdcal, openpyxl, six, python-dateutil, dbf, csvkit
  Running setup.py install for gdal ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-56Rm7O/gdal/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-KGJm8q-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    copying gdal.py -> build/lib.linux-x86_64-2.7
    copying ogr.py -> build/lib.linux-x86_64-2.7
    copying osr.py -> build/lib.linux-x86_64-2.7
    copying gdalconst.py -> build/lib.linux-x86_64-2.7
    copying gnm.py -> build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdal.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/osr.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/__init__.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdalnumeric.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gnm.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdalconst.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdal_array.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/ogr.py -> build/lib.linux-x86_64-2.7/osgeo
    running build_ext
    building 'osgeo._gdal' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/extensions
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I../../ogr/ogrsf_frmts -I../../gnm -I../../apps -I/usr/include/python2.7 -I. -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    extensions/gdal_wrap.cpp:3168:22: fatal error: cpl_port.h: No such file or directory
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-56Rm7O/gdal/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-KGJm8q-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-56Rm7O/gdal/

csvkit failure

$ pip install csvkit==0.9.0
Collecting csvkit==0.9.0
  Downloading https://files.pythonhosted.org/packages/37/39/e8a2bd3fe4c132fb0a09f7e8382341c4aeda2144506518465eccb4ffa655/csvkit-0.9.0.tar.gz
Collecting xlrd>=0.7.1 (from csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/a6/0c/c2a72d51fe56e08a08acc85d13013558a2d793028ae7385448a6ccdfae64/xlrd-2.0.1-py2.py3-none-any.whl (96kB)
    100% |████████████████████████████████| 102kB 8.8MB/s 
Collecting sqlalchemy>=0.6.6 (from csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/e0/24/75f1f1464457e9c01b138145636997020133566282df710a41a3cfdcec76/SQLAlchemy-1.4.9-cp27-cp27mu-manylinux1_x86_64.whl (1.5MB)
    100% |████████████████████████████████| 1.5MB 840kB/s 
Collecting openpyxl>=2.0.3 (from csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/39/08/595298c9b7ced75e7d23be3e7596459980d63bc35112ca765ceccafbe9a4/openpyxl-3.0.7-py2.py3-none-any.whl (243kB)
    100% |████████████████████████████████| 245kB 5.3MB/s 
Collecting six>=1.6.1 (from csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Collecting python-dateutil==2.2 (from csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/75/c5/85d027471fa665f8c8b8eb0b925f9d84b4eee745a257b16de4957de99e81/python-dateutil-2.2.tar.gz (259kB)
    100% |████████████████████████████████| 266kB 4.8MB/s 
Collecting dbf==0.94.003 (from csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/7c/ef/79873517838910e8f1bd5610ab3de380a3886675103072727266eec91bbc/dbf-0.94.003.tar.gz (79kB)
    100% |████████████████████████████████| 81kB 10.3MB/s 
Collecting importlib-metadata; python_version < "3.8" (from sqlalchemy>=0.6.6->csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/9a/5a/5bee793e43f6499b7e28de7c0333f49752590826b08a456f3874f555f4f5/importlib_metadata-4.0.0.tar.gz (41kB)
    100% |████████████████████████████████| 51kB 11.5MB/s 
  Running setup.py (path:/tmp/pip-build-Co1hk3/importlib-metadata/setup.py) egg_info for package importlib-metadata produced metadata for project name unknown. Fix your #egg=importlib-metadata fragments.
Collecting et-xmlfile (from openpyxl>=2.0.3->csvkit==0.9.0)
  Downloading https://files.pythonhosted.org/packages/22/28/a99c42aea746e18382ad9fb36f64c1c1f04216f41797f2f0fa567da11388/et_xmlfile-1.0.1.tar.gz
Building wheels for collected packages: csvkit, python-dateutil, dbf, unknown, et-xmlfile, unknown
  Running setup.py bdist_wheel for csvkit ... done
  Stored in directory: /home/ubuntu/.cache/pip/wheels/b1/e6/5e/12a42a3bc016cf19674eec242ff08479bcd84377270bb7f43d
  Running setup.py bdist_wheel for python-dateutil ... done
  Stored in directory: /home/ubuntu/.cache/pip/wheels/97/40/49/e02356db91efa7a5da28ec2e5c61f0fb2c0aa21bbd5da95805
  Running setup.py bdist_wheel for dbf ... done
  Stored in directory: /home/ubuntu/.cache/pip/wheels/fa/fb/ec/4da45ba6853875f5adc64cb3cc0fccc799f739ed4e7584a082
  Running setup.py bdist_wheel for unknown ... done
  Stored in directory: /home/ubuntu/.cache/pip/wheels/d4/24/c8/3783bc6cc33cace61d33e1b6a76659800cdf3d2bf2bc7a1c42
  Running setup.py bdist_wheel for et-xmlfile ... done
  Stored in directory: /home/ubuntu/.cache/pip/wheels/2a/77/35/0da0965a057698121fc7d8c5a7a9955cdbfb3cc4e2423cad39
  Running setup.py bdist_wheel for unknown ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-Co1hk3/unknown/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpkSZCFhpip-wheel- --python-tag cp27:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  IOError: [Errno 2] No such file or directory: '/tmp/pip-build-Co1hk3/unknown/setup.py'
  
  ----------------------------------------
  Failed building wheel for unknown
  Running setup.py clean for unknown
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-Co1hk3/unknown/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" clean --all:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  IOError: [Errno 2] No such file or directory: '/tmp/pip-build-Co1hk3/unknown/setup.py'
  
  ----------------------------------------
  Failed cleaning build dir for unknown
Successfully built csvkit python-dateutil dbf unknown et-xmlfile
Failed to build unknown
Installing collected packages: xlrd, unknown, sqlalchemy, et-xmlfile, openpyxl, six, python-dateutil, dbf, csvkit
Successfully installed csvkit-0.9.0 dbf-0.94.3 et-xmlfile-1.0.1 openpyxl-3.0.7 python-dateutil-2.2 six-1.15.0 sqlalchemy-1.4.9 unknown-0.0.0 xlrd-2.0.1
You are using pip version 8.1.1, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Expected result

Successful compilation of dependencies

$ sudo pip install --no-use-wheel -r python_requirements.txt
DEPRECATION: --no-use-wheel is deprecated and will be removed in the future.  Please use --no-binary :all: instead.
The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): chardet==2.3.0 in /home/ubuntu/.local/lib/python2.7/site-packages (from -r python_requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): argparse==1.2.1 in /usr/lib/python2.7 (from -r python_requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): brewery==0.6 in /home/ubuntu/.local/lib/python2.7/site-packages (from -r python_requirements.txt (line 3))
Requirement already satisfied (use --upgrade to upgrade): redis==2.4.9 in /home/ubuntu/.local/lib/python2.7/site-packages (from -r python_requirements.txt (line 4))
Requirement already satisfied (use --upgrade to upgrade): python-varnish==0.1.2 in /home/ubuntu/.local/lib/python2.7/site-packages (from -r python_requirements.txt (line 5))
Requirement already satisfied (use --upgrade to upgrade): gdal==2.2.2 in /home/ubuntu/.local/lib/python2.7/site-packages (from -r python_requirements.txt (line 6))
Requirement already satisfied (use --upgrade to upgrade): csvkit in /home/ubuntu/.local/lib/python2.7/site-packages (from -r python_requirements.txt (line 7))
Collecting openpyxl==2.1.3 (from -r python_requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/86/7a/bba821c554b5fbde20aaa70cf5ae8c284883f38162deee7852e068db6246/openpyxl-2.1.3.tar.gz (96kB)
    100% |████████████████████████████████| 102kB 7.9MB/s 
Requirement already satisfied (use --upgrade to upgrade): xlrd>=0.7.1 in /home/ubuntu/.local/lib/python2.7/site-packages (from csvkit->-r python_requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): python-dateutil==2.2 in /home/ubuntu/.local/lib/python2.7/site-packages (from csvkit->-r python_requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): six>=1.6.1 in /home/ubuntu/.local/lib/python2.7/site-packages (from csvkit->-r python_requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): sqlalchemy>=0.6.6 in /home/ubuntu/.local/lib/python2.7/site-packages (from csvkit->-r python_requirements.txt (line 7))
Requirement already satisfied (use --upgrade to upgrade): dbf==0.94.003 in /home/ubuntu/.local/lib/python2.7/site-packages (from csvkit->-r python_requirements.txt (line 7))
Collecting jdcal (from openpyxl==2.1.3->-r python_requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/7b/b0/fa20fce23e9c3b55b640e629cb5edf32a85e6af3cf7af599940eb0c753fe/jdcal-1.4.1.tar.gz
Collecting importlib-metadata; python_version < "3.8" (from sqlalchemy>=0.6.6->csvkit->-r python_requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/9a/5a/5bee793e43f6499b7e28de7c0333f49752590826b08a456f3874f555f4f5/importlib_metadata-4.0.0.tar.gz (41kB)
    100% |████████████████████████████████| 51kB 11.8MB/s 
  Running setup.py (path:/tmp/pip-build-YG4Uph/importlib-metadata/setup.py) egg_info for package importlib-metadata produced metadata for project name unknown. Fix your #egg=importlib-metadata fragments.
  Requirement already satisfied (use --upgrade to upgrade): unknown from https://files.pythonhosted.org/packages/9a/5a/5bee793e43f6499b7e28de7c0333f49752590826b08a456f3874f555f4f5/importlib_metadata-4.0.0.tar.gz#sha256=2e881981c9748d7282b374b68e759c87745c25427b67ecf0cc67fb6637a1bff9 in /home/ubuntu/.local/lib/python2.7/site-packages (from sqlalchemy>=0.6.6->csvkit->-r python_requirements.txt (line 7))
Installing collected packages: jdcal, openpyxl
  Running setup.py install for jdcal ... done
  Found existing installation: openpyxl 3.0.7
    Uninstalling openpyxl-3.0.7:
      Successfully uninstalled openpyxl-3.0.7
  Running setup.py install for openpyxl ... done
Successfully installed jdcal-1.4.1 openpyxl-2.1.3
You are using pip version 8.1.1, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gabegm