Task #22894
Feature #22889: Build spack_to_ups converter
Initial coding
Start date:
07/10/2019
Due date:
% Done:
100%
Estimated time:
History
#1 Updated by Marc Mengel 4 months ago
So at the moment, pursuing this as actually two module types: ups_table and ups_version to write the version and table files respectively.
Currently on the ups_gen branch of the spack repository in this project.
#2 Updated by Marc Mengel 4 months ago
So currently, duplicated the dotkit modules files for ups_table and ups_version, and added them to the list of module types, but it blows up partway there.
So Trying to figure out how it all gets called, I injected an exception and a stack trace print into the code where it loads the jinja templates for the existing module types, and did an install, it looks like:
File "/scratch/mengel/spack/bin/spack", line 48, in <module>
sys.exit(spack.main.main())
File "/home/mengel/spack/lib/spack/spack/main.py", line 696, in main
return _invoke_command(command, parser, args, unknown)
File "/home/mengel/spack/lib/spack/spack/main.py", line 446, in _invoke_command
return_val = command(parser, args)
File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 330, in install
install_spec(args, kwargs, abstract, concrete)
File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 205, in install_spec
install(spec, kwargs)
File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 194, in install
spec.package.do_install(**kwargs)
File "/home/mengel/spack/lib/spack/spack/package.py", line 1618, in do_install
self, build_process, dirty=dirty, fake=fake)
File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 810, in fork
p.start()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 130, in start
self._popen = Popen(self)
File "/usr/lib64/python2.7/multiprocessing/forking.py", line 126, in __init__
code = process_obj._bootstrap()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 765, in child_process
return_value = function()
File "/home/mengel/spack/lib/spack/spack/package.py", line 1578, in build_process
spack.hooks.post_install(self.spec)
File "/home/mengel/spack/lib/spack/spack/hooks/__init__.py", line 54, in __call__
hook(*args, **kwargs)
File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 29, in <lambda>
post_install = lambda spec: _for_each_enabled(spec, 'write')
File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 22, in _for_each_enabled
getattr(generator, method_name)()
File "/home/mengel/spack/lib/spack/spack/modules/common.py", line 726, in write
traceback.print_stack()
File "/scratch/mengel/spack/bin/spack", line 48, in <module>
sys.exit(spack.main.main())
File "/home/mengel/spack/lib/spack/spack/main.py", line 696, in main
return _invoke_command(command, parser, args, unknown)
File "/home/mengel/spack/lib/spack/spack/main.py", line 446, in _invoke_command
return_val = command(parser, args)
File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 330, in install
install_spec(args, kwargs, abstract, concrete)
File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 205, in install_spec
install(spec, kwargs)
File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 194, in install
spec.package.do_install(**kwargs)
File "/home/mengel/spack/lib/spack/spack/package.py", line 1618, in do_install
self, build_process, dirty=dirty, fake=fake)
File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 810, in fork
p.start()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 130, in start
self._popen = Popen(self)
File "/usr/lib64/python2.7/multiprocessing/forking.py", line 126, in __init__
code = process_obj._bootstrap()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 765, in child_process
return_value = function()
File "/home/mengel/spack/lib/spack/spack/package.py", line 1578, in build_process
spack.hooks.post_install(self.spec)
File "/home/mengel/spack/lib/spack/spack/hooks/__init__.py", line 54, in __call__
hook(*args, **kwargs)
File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 29, in <lambda>
post_install = lambda spec: _for_each_enabled(spec, 'write')
File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 22, in _for_each_enabled
getattr(generator, method_name)()
File "/home/mengel/spack/lib/spack/spack/modules/common.py", line 726, in write
traceback.print_stack()
#3 Updated by Marc Mengel 4 months ago
- Tracker changed from Feature to Task
#4 Updated by Marc Mengel 4 months ago
- % Done changed from 30 to 60
Okay, so now we generate table and version files, getting something resembling a UPS database.
https://cdcvs.fnal.gov/redmine/projects/spack-infrastructure/repository/spack/diff?utf8=%E2%9C%93&rev=72a1368fc866b990a1bde11c617d79897d4e252f&rev_to=030f0951d4ff888766145667a7d76dc1c21cfa88
So remaining bits to get right are:
- generating ups flavor and qualifiers from spack info
- finding dependencies, as they don't normally put dependencies in their module files, rather they want you to use "spack load -r" to do recursive setups.
#5 Updated by Marc Mengel 4 months ago
Some more progress, flavor translation in version file but need the same for dependencies.
after lookining in spec._cmp_key_cache to find out specifics.
Also need suitable qualifiers in table file(?) and/or table file per instance
commit:be5ba62
#6 Updated by Marc Mengel about 2 months ago
- % Done changed from 60 to 100