Support efforts to move LE Linac LLRF to kernel 6.4
A second LLRF development station has been put together, mostly for training purposes for Trevor Butler & controls department Pierrick Hanlet.
Currently, both LE linac development stations are running kernel 5.5.
Trevor and Pierrick are working out the bugs and learning the LLRF system in the process. When the bugs are worked out, the plan is to move the kernel to 6.4 and test.
#1 Updated by Ed Cullerton almost 2 years ago
- % Done changed from 0 to 80
- Status changed from New to Assigned
Two development stations have been made to work(fully functional) with the VXworks 5.5 Kernel.
Pierrick is currently attempting to bring one of the stations up to the 6.4 kernel and is having some issues. I have been supporting his efforts as much as I can, but my experience is limited. We may need some help from experts who have more experience with this system.
#4 Updated by Ed Cullerton 9 months ago
Both system are currently up and running, and we are waiting for the 6.4 system to stop working and hang. It may take over a day for the system to hang.
A scope is currently monitoring the backplane trigger, so we will be able to see if the backplane trigger disappears when the system hangs.
#5 Updated by Ed Cullerton 8 months ago
The system eventually hung up after a day of running. The trigger on the backplane was no longer present (observed with a scope). This indicates that the trigger is not being transmitted, or it is not being received, on the backplane of the VXI crate.
The MFC board in the 6.4 station was replaced with the known working MFC board from the 5.5 station, and the test was repeated. The same failure occurred using the known working MFC board. Will test one more time to see if the system hangs again.
#6 Updated by Ed Cullerton 8 months ago
The 5.5 station (LLRF0) was turned off to check for crosstalk issues. The system hung up sometime after 50 hrs.
The next test is to run the 6.4 card(LLRF6) in the lower crate.
These are some remaining explanations for losing the trigger in the hardware:
1. We lose the FPGA clock (805 MHz). Maybe a bad cable or connector could cause this. Both systems run off of the same 805 MHz signal, so if all the cables/connectors are good, a loss in clock would be seen on both systems. If we have a problem a cable/connection to the top crate, the the next test could rule this out. (running the 6.4 card (LLRF6) in the lower crate)
2. The VXI crate is broke. Next test could rule this out.
3. The kinetic systems adapter is not configured properly. There are dip switches on the hardware that should match on both system. This could be easily verified.
4. The kinetic systems adapter is broke. We could rule this out if we run the 5.5 station (LLRF0) at 6.4 and see the same problem.
The remaining software possibilities(kinetic systems driver) can be explored again if we rule out posiible hardware problems.
#7 Updated by Ed Cullerton 8 months ago
I moved the 5500 card(LLRF6) to the bottom VXI crate this morning. In the process, I found that the 805 clock was disconnected. I don’t know when this happened, but it may be the cause of problem.
The card is currently running in the bottom VXI crate, so let’s see what happens. If all good, I’ll move it back up the top crate, make sure the 805 is connected, and test again.
#8 Updated by Ed Cullerton 8 months ago
Moving the 6.4 LLRF6 slot zero card into the bottom LLRF0 VXI crate resulted in the same trigger issue.
For the next test, both LLRF0 and LLRF6 have been updated to 6.4. Both system are running in their original crate (LLRF0 in bottom VXI, LLRF6 in top VXI).
If both system have the trigger issues after some time, I think we can conclude that it is a software issue.
If LLRF0 works OK, and LLRF6 has the trigger issue after some time, I think we can conclude there is a problem with the MVME5500+KSVXI adapter hardware.
#9 Updated by Ed Cullerton 8 months ago
With both system running with 6.4, they both failed with the same trigger issue. With this, we can probably say that it is not a hardware problem.
We are in the process of figuring out a way forward debugging software. We do not yet know if the systems failed at the same time, but this would be helpful information.
#11 Updated by Ed Cullerton 7 months ago
The 5500 board that Dennis gave us has been tested and is currently running in LLRF0 (LIP740).
We now have two spare 5500 boards loaded with the 5.5 kernel and tested. The third spare board is currently in LLRF6 (lip746) and is being used for continued testing of the 6.4 kernel.
#12 Updated by Ed Cullerton 2 months ago
After a long effort, it has been decided to not pursue upgrading the kernel to 6.4, and remain at 5.5.
The development stations (lip740 & lip 746) are both up and running 5.5.
We will request that the 5.5 development computer remain on site in case it is needed in an emergency situation.
Attached are the last boot records for both development stations