Task #22894
Feature #22889: Build spack_to_ups converter
Initial coding
100%
History
#1 Updated by Marc Mengel over 1 year ago
So at the moment, pursuing this as actually two module types: ups_table and ups_version to write the version and table files respectively.
Currently on the ups_gen branch of the spack repository in this project.
#2 Updated by Marc Mengel over 1 year ago
So currently, duplicated the dotkit modules files for ups_table and ups_version, and added them to the list of module types, but it blows up partway there.
So Trying to figure out how it all gets called, I injected an exception and a stack trace print into the code where it loads the jinja templates for the existing module types, and did an install, it looks like:
File "/scratch/mengel/spack/bin/spack", line 48, in <module> sys.exit(spack.main.main()) File "/home/mengel/spack/lib/spack/spack/main.py", line 696, in main return _invoke_command(command, parser, args, unknown) File "/home/mengel/spack/lib/spack/spack/main.py", line 446, in _invoke_command return_val = command(parser, args) File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 330, in install install_spec(args, kwargs, abstract, concrete) File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 205, in install_spec install(spec, kwargs) File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 194, in install spec.package.do_install(**kwargs) File "/home/mengel/spack/lib/spack/spack/package.py", line 1618, in do_install self, build_process, dirty=dirty, fake=fake) File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 810, in fork p.start() File "/usr/lib64/python2.7/multiprocessing/process.py", line 130, in start self._popen = Popen(self) File "/usr/lib64/python2.7/multiprocessing/forking.py", line 126, in __init__ code = process_obj._bootstrap() File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 765, in child_process return_value = function() File "/home/mengel/spack/lib/spack/spack/package.py", line 1578, in build_process spack.hooks.post_install(self.spec) File "/home/mengel/spack/lib/spack/spack/hooks/__init__.py", line 54, in __call__ hook(*args, **kwargs) File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 29, in <lambda> post_install = lambda spec: _for_each_enabled(spec, 'write') File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 22, in _for_each_enabled getattr(generator, method_name)() File "/home/mengel/spack/lib/spack/spack/modules/common.py", line 726, in write traceback.print_stack() File "/scratch/mengel/spack/bin/spack", line 48, in <module> sys.exit(spack.main.main()) File "/home/mengel/spack/lib/spack/spack/main.py", line 696, in main return _invoke_command(command, parser, args, unknown) File "/home/mengel/spack/lib/spack/spack/main.py", line 446, in _invoke_command return_val = command(parser, args) File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 330, in install install_spec(args, kwargs, abstract, concrete) File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 205, in install_spec install(spec, kwargs) File "/home/mengel/spack/lib/spack/spack/cmd/install.py", line 194, in install spec.package.do_install(**kwargs) File "/home/mengel/spack/lib/spack/spack/package.py", line 1618, in do_install self, build_process, dirty=dirty, fake=fake) File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 810, in fork p.start() File "/usr/lib64/python2.7/multiprocessing/process.py", line 130, in start self._popen = Popen(self) File "/usr/lib64/python2.7/multiprocessing/forking.py", line 126, in __init__ code = process_obj._bootstrap() File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/home/mengel/spack/lib/spack/spack/build_environment.py", line 765, in child_process return_value = function() File "/home/mengel/spack/lib/spack/spack/package.py", line 1578, in build_process spack.hooks.post_install(self.spec) File "/home/mengel/spack/lib/spack/spack/hooks/__init__.py", line 54, in __call__ hook(*args, **kwargs) File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 29, in <lambda> post_install = lambda spec: _for_each_enabled(spec, 'write') File "/home/mengel/spack/lib/spack/spack/hooks/module_file_generation.py", line 22, in _for_each_enabled getattr(generator, method_name)() File "/home/mengel/spack/lib/spack/spack/modules/common.py", line 726, in write traceback.print_stack()
#3 Updated by Marc Mengel over 1 year ago
- Tracker changed from Feature to Task
#4 Updated by Marc Mengel over 1 year ago
- % Done changed from 30 to 60
Okay, so now we generate table and version files, getting something resembling a UPS database.
https://cdcvs.fnal.gov/redmine/projects/spack-infrastructure/repository/spack/diff?utf8=%E2%9C%93&rev=72a1368fc866b990a1bde11c617d79897d4e252f&rev_to=030f0951d4ff888766145667a7d76dc1c21cfa88
So remaining bits to get right are:
- generating ups flavor and qualifiers from spack info
- finding dependencies, as they don't normally put dependencies in their module files, rather they want you to use "spack load -r" to do recursive setups.
#5 Updated by Marc Mengel over 1 year ago
Some more progress, flavor translation in version file but need the same for dependencies.
after lookining in spec._cmp_key_cache to find out specifics.
Also need suitable qualifiers in table file(?) and/or table file per instance
commit:be5ba62
#6 Updated by Marc Mengel over 1 year ago
- % Done changed from 60 to 100