OSG: Create plug-in architecture for frontends
Request from HCC. We should discuss if we want to support this and how.
This is a follow-up email from our discussion on Tuesday.
Right now, when we develop a plugin (for example, a new file transfer plugin for Condor that is to be deployed with the pilots), packaging and deployment on other sites is quite complex. It is easy for an RPM to "dump the binaries" to the right location, but integration with the VO frontend requires the admin to edit the configuration. In our experience, each install requires several back-and-forth emails with the plugin developers - it takes a lot of time and is error-prone.
I'd like to request that glideinWMS makes it easier for external developers to make plugins for the VO frontend, deployed via RPM.
While I'm open to many solutions, here's a suggested implementation:
1) Have a "plugins.d" directory somewhat equivalent to Condor's "config.d". On reconfig, the VO frontend will process each file in this directory (so, an RPM can install a new plugin by dropping a file here - but admins can override or disable the RPM by editing the file).
2) Each plugin file is an XSLT transform. The transform takes the current frontend XML file and does an arbitrary transform on it.
- For example, add the appropriate XML configuration stanzas to change the Condor configuration, add a script to stagein, etc, to each configured group.
3) Once all configurations are processed, the reconfig will utilize the resulting frontend.xml file instead of the original one. Failure of any of the plugins is a failure of the reconfig command.
This way, an RPM install can automatically integrate with the VO frontend configuration, but admins can override it for complex setups.
#1 Updated by Douglas Strain over 8 years ago
From what I understand, this would basically be scrapping the reconfig/upgrade process that we have, as well as basically the whole creation/lib directory. We would replace it by a directory of XSLT plugins (plus maybe a few custom functions if XSLT was not quite up the more complicated task(s) of creating condor tarballs, etc). For instance, one XSLT to create the descript file(s), one to create certmaps, one to create/modify condor_config, one to modify/merge frontend.xml, etc etc.
This sounds like quite a bit of work, but I think it would be worth it. That section of code is extremely difficult to work with and quite confusing as to what it does. This proposal (or one close to it) would make it a lot cleaner, more standard, more straightforward, and a lot more customizable. (For instance, a new xml parameter would just need an entry in the XSLT to pop it into a descript or into a condor config).
I'd be interested in others opinions as well as a conversation with Brian to make sure that we understand his use case.
#2 Updated by Parag Mhashilkar over 8 years ago
I think the use case is more specific to condor's file transfer plugin. Currently plugins are integrated in the setup as follows -
- Transfer Plugins and all the required setup scripts are treated as independent custom scripts
- The way I wrote the Globus Online plugin which most of the adopters picked up from have atleast 2 scripts
- Transfer protocol client in native format. Optional and not required if client is deployed on the worker node. * Transfer plugin scripts that uses the client to do the transfer and is registered with the condor of the glidein as a FILETRANSFER_PLUGINS * Setup script that deploys the client and sets up condor_config by making sure FILETRANSFER_PLUGINS is set along with any other info plugin needs to use the client
My understanding of what Brian is asking is a native and easy means to handle these tasks
- He wants us to support either an extension to the <files> tag or a new tag that handles plugin files
- Properties of plugin files are listed in a xml file (as per his description) and will give the reconfig more instructions on how to configure the plugin in frontend/factory. For example, the xml file can point to 2 or more files describing my sample setup above. The config file need not be xml and can be in a simple attribute = value format
- Setup script * Plugin script * Enable/Disable tag for admins to override, just like yum repos * Zero or more client installers
- On reconfig, reconfig task will use these plugin configs file to get more info and add required <files> to the frontend.xml and use this new frontend.xml for actual reconfig. I am not in favor of this intermediate frontend.xml. We can get around this by natively supporting plugins.d dir type in <file> same way we do for a tar file. Or Alternatively, we just create <plugins> tag at same level as <files>
If this is what HCC wants, this is much isolated problem than what I initially thought and should be relatively easy. I should have done it initially when I wrote the GO plugin but never had a use case to justify the work.
#3 Updated by Brian Bockelman over 8 years ago
(Sorry for the late response, I didn't originally have permission to post comments.)
I used the example of the file transfer plugin as a motivation for the wider problem. We have no way to programmatically change / alter defaults, or have new packages add functionality while maintaining the end-user's ability to override whatever we provide.
Other than the example of file transfer plugins, consider what we would have to do to add a new OSG factory. Or what we will need to do to change all the DNs next year (DOEgrids is going away). It would be nice if we could distribute the factory endpoints via RPM, and allow sites to take those if they want.
Doug - the reason I suggested XSLT was not due to some love of that particularly difficult language, but the fact that it can take in XML and spit out XML. It can be applied as a pre-processing step when "reconfig" is executed, so gWMS internals would not see any of this. No tarball or directory creation needs to change.
While it might be quite interesting to redesign the configuration language (and, I think, needed), I'm not sure if this is what Burt had in mind.
#9 Updated by Brian Bockelman over 7 years ago
Ok, here's a simple silly XSLT transform which adds a comment to any <security/> tag. I'm going to work with Dan to do something more substantial.
I did have to apply the following patch:
--- a/creation/lib/cWParams.py +++ b/creation/lib/cWParams.py @@ -174,8 +174,11 @@ class Params: if argv=="-help": raise RuntimeError,"\nA config file will contain:\n%s\n\nThe config file will be in XML format."%self.get_description(" ") - - self.cfg_name=os.path.abspath(argv) + if hasattr(argv, "name"): + self.cfg_name=os.path.abspath(argv.name) + else: + self.cfg_name=os.path.abspath(argv) + #print open(self.cfg_name).read() self.load_file(self.cfg_name) self.subparams.validate(self.defaults,self.get_top_element())
Probably something in the XSLT transform code is feeding cWParams.py a file descriptor where a file name is expected; I don't think we should apply the patch as-is, but figure out what's feeding the incorrect parameter.
#14 Updated by Parag Mhashilkar about 7 years ago
- Status changed from Feedback to Assigned
- Assignee changed from Parag Mhashilkar to Burt Holzman
Changes look ok. I did not test them though. Also, we need some documentation with examples. I can write the docs but atleast need sample usage etc to get started. Assigning back to you.