|
Lego_NXT module
John Davis [71 posts] |
9 year
|
With the new EV3 out, I now have a Lego NXT that can be repurposed. So I built a Lego robot arm with it. The idea is to let the Windows 7 Pro PC-based RoboRealm track a ball and direct the arm towards it. (We have a pick-up-the-orange-golf-ball contest in November.)
Yesterday I could insert the Lego_NXT module into a blank RR program and manually control the NXT motors. (With Bluetooth, no less!)
Today nothing worked, neither Bluetooth or USB, until after several restarts. (RR also froze once and had to be killed.) The only thing that is the same is usually the "Remember as default" reverts to COM1, no matter how often I select the USB port or the correct Bluetooth port.
I tried again the last few minutes; now the manual interface doesn't work anymore. My RR is version 2.62.2.
Update: I restarted RR and manual does work again. It is not clear whether motors should be enabled or not for program mode; the ball picker example seems to have them enabled.
My original RoboMagellan RR program still works perfectly and tracks the golf ball.
I have not had any simple RR program work for Lego_NXT yet. I tried a two-line RR program with just two Lego_NXT modules. (A more elaborate RR program with one-time initialization and If_Statement did not seem to do anything either.)
The programs on the NXT are "Up" and "Down". The first time editing, when I entered "Up" in the file startup box (Start Lego Program->Filename) for Lego_NXT, the arm raised itself perfectly. Nothing else has worked since, not even deleting the entry and re-entering it.
At least now the USB default seems to be sticky. Since the NXT is mounted on the truck which carries the PC, I don't need Bluetooth.
What I would like to see is an example that shows RR executing a program (filename) exactly once on the NXT. The example should show exactly how to edit the Lego_NXT module to do that.
If that is too complicated, RR executing two programs over and over would help. I can stop the Up-Down-Up-Down sequence before the motors wear out.
|
|
|
|
John Davis [71 posts] |
9 year
|
|
From lack of replies, I assume there is not much interest in the RR-NXT interface. So I will abandon that effort and look for a simple inexpensive gripper with servos that my Pololu Micro-Maestro can control.
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
Actually, its not through lack of interest but more that we don't typically have direct access to the NXT. Since we don't use the system on a daily basis we loan out our NXT's to a local organization (DeepSpace in Colorado, Parker) that teaches programming and robotics to kids. So when a request comes up it takes a bit to get access to that hardware.
I've attached a program that cycles every 5 seconds from DemoV2 to program to program2. You should be able to adapt that to your own programs. Note, we discovered that without a delay aborting one app to run another required about a 1 second delay. This was just added to the latest RR rev in case you are killing a running program before it aborts normally. Note that there isn't any change to the NXT program on the device as the PC interface will just kill it after a number of seconds. Hopefully you instead can provide a sensor trigger to instead run the next program ... but time can be used in a last case scenario.
STeven.
 program.robo
|
|
|
|
John Davis [71 posts] |
9 year
|
Thanks! I simplified your VBScript to cycle just two programs, Up and Down. It works perfectly when RR program and NXT brick boot up.
However, ...
When I release the "Run" button, create an identical new program ("New" button), duplicate the VBScript and Lego_NXT settings, and then press "Run" again, an Error box pops up with "Could not initialize on port USB!".
I have been through many permutations of this, with identical and non-identical programs, editing and re-editing. After a while, RR just freezes and has to be killed with Task Manager.
The only procedure that seems to work reliably is to restart both RR and NXT. Rebooting just the NXT seems to work most of the time. When RR hangs, Task Manager shows it using quite a bit of CPU time; probably looping, waiting for a semaphore that will never change again.
The above tests all use the "Start Lego Program -> Variable" field in the Lego_NXT edit dialog. If I use the "Filename" field instead, it is hard to get anything to work. Some days the arm will move once or twice, but not today.
John
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
I think we fixed this issue. It seems that the Fantom.dll doesn't actually stop or unload the connection even when you request the connection be destroyed ... so instead we just keep it in a state that can be reused. Seems to work better now.
In terms of what you are trying to do, I'd not use the Filename field but instead the variable field. The filename field is used only when you want to run a single program on the NXT that you would then communicate to. Since you want to run multiple files you need the variable in order to change the value. Note, you use the Filename OR the variable but not both.
See the latest version for this update.
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
Thanks again! It is getting better, but there is still some odd behavior.
First, changes to the VBS script seem to be one version behind, even when editing directly with RR. Seemed to be the same after switching to external editor. Maybe I was supposed to save the RR program to disk also. I haven't investigated this methodically.
Second, in order to get the attached RR program to execute the CaptureBall program on the NXT exactly once, I had to set the program_to_run variable at the head of the RR program every iteration of the pipeline; if I set it with the Once Only box checked, the NXT program never ran.
Notes:
Rubber bands ensure that the claw remains closed after the NXT program finishes; RR then will take over and steer the robot with ball back to the collection box.
RR starts with the claw closed (unpowered) and the arm raised (unpowered). The CaptureBall sequence is open, down, close, up, and end NXT program.
program.robo
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
VBScript changes - What leads you to believe it is one version behind? If you make a change to the script and then press the reload button do you need to do that twice before you see the changes? We've not been able to replicate this exact behavior. I wonder if the state machine is causing confusion since the code will only run if the state is in the right mode.
Once Execution - The problem you were having is you needed one additional state at the end to set the program to null which would effectively stop the pipeline. The problem then is how to start the loop again ... we added in a button to take care of that.
If you look at the attached, its cleaned up a bit by adding in that last termination state that halts the cycle ... until a press of the Green Run button sets the state back to zero. See if this works better for you.
STeven.
program.robo
|
|
|
|
John Davis [71 posts] |
9 year
|
Restarting the pipeline with an extra Run button does seem to make VBScript changes reload; I did not know reloading was necessary, just assumed it would reload when the IDE Run was reset.
The JO ("Just Once") problem is still unclear to me. You added a last line to my simple program which is now like this:
1 Set program_to_run = CaptureBall
2 Camera Properties
3 Lego_NXT
4 Display_Variables
5 (disabled) VBScript
6 Set program_to_run = null
Relevant parameters are:
SO Line 1 - Set Once (T/F)
RO RR IDE Run On? (T/F)
NP Variable "program_to_run" for NXT's on-brick program
PF RR program freshly loaded from hard drive? (T/F)
Sequence 1: RO=>F, PF=>T, SO==F<\b>, RO=>T; robot performs capture exactly once, pipeline continues to run, NP display remains set to CaptureBall (CB) as I would expect. RO=>F, RO=>T does nothing; robot never makes a move, NP stays CB.
Sequence 2: RO=>F, PF=>T, SO=>T<\b>, RO=>T; robot closes claw, but arm remains up. NP now null as it is supposed to be after first iteration. Recycling RO, as before, changes nothing.
|
|
|
|
John Davis [71 posts] |
9 year
|
|
Sorry about the excess bold type; I thought <\b> would end it and I failed to look up an HTML reference.
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
Actually I made more changes than that ... I checked the post and the right robofile is posted. Can you confirm that what you see in the pipeline is what I took a snapshot below? Its possible that when you downloaded the file it either didn't overwrite what you previously had.
STeven.
program.robo
|
|
|
|
John Davis [71 posts] |
9 year
|
Sorry, my reference was to what you get when you download the program I posted. Your program method with the green start button works, but won't be useful for competition when the robot has to be fully autonomous after the initial start.
Anyway, the 6-line program listed in my previous post is just too simple to be wrong. :) I am not an expert but I have snapped together computer vision pipelines using OpenCV, VTK, and even C++. My guess is there is some kind of RR internal race condition where the script continues past Set Once, fails to execute the Lego_NXT module properly, but keeps on going - thus guaranteeing that program_to_run will thereafter be null and that the NXT will never execute anything again. If Set Once is off, the Lego_NXT module is eventually awakened, executes once - then for some reason never runs again.
The behavior of the RR Run button must be deliberate not to re-initialize, but rather act as a Pause button. Is reloading the RR program the only way to start fresh?
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
What would the condition to restart be? The button is just for testing purposes. The system should run exactly once on startup ... I added the Run button just to allow you to retest without having to restart. Perhaps I'm missing something ... or perhaps we need to add something that receives the start signal from the NXT? I imagine when the contest is started something is pressed or signaled? Any idea on what that would be?
So we can trigger it whenever you need but I'm just not sure what that trigger condition is?
Keep in mind that each module executes after the other in sequence. The VBScript module will only allow the next module to run when it has completed ... same with the Lego NXT. So modules will not be skipped but may appear to be so given the nature of the information. We avoided many race conditions using this structure which for image processing makes sense cause you cannot skip processing modules otherwise the next one would not work correctly.
For example, setting the variable to null at the end and then again at the start to some other value at the start of the pipeline doesn't do anything. One will overwrite the other even before anything reacts to that variable change. I.e. think of the pipeline being in a while loop. Setting something at the end and then at the beginning doesn't have an effect. Because everything is sequential, the NXT module will only see and react to the variable when it executes. Thus you can change the variable a million times and the NXT module will never know at it will only see the value of the variable when it executes.
Hope that helps build the model that you are working with. Its not like many other systems where you do need to worry about race conditions.
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
You wrote
"For example, setting the variable to null at the end and then again at the start to some other value at the start of the pipeline doesn't do anything. One will overwrite the other even before anything reacts to that variable change. I.e. think of the pipeline being in a while loop. Setting something at the end and then at the beginning doesn't have an effect."
I completely understand this behavior when the Set Once box is not checked in the Set module. However, when Set Once is set, then I expect (for that variable) the Set module to be like the init block in an Arduino sketch; that variable will not be set again each pass through the loop. That feature is in fact working correctly, as far as I can tell.
So I now believe the Lego_NXT module is returning before it completes correctly.
I have a suggestion. If you can borrow an NXT and two motors with cables, you could simulate my model without actually constructing the arm; just lay the motors on the table. The arm is on port A, the claw on B.
The NXT program, CaptureBall.rbt, is attached, also a picture of the top level graphic, CaptureBall.jpg.
FYI, I intend to eventually use your impressive State Machine module for competition. There will be ample control for triggers. I just need to figure out exactly how to use vision to decide when to change states. The robot will need to travel a few meters tracking an neon red golf ball, then position itself precisely before invoking the NXT capture routine. Once it has the ball, it has to turn around and find the neon red cross on the deposit box. Once in position at the box, it then will run another NXT routine which lowers the arm and opens the claw.
I really appreciate the effort that has gone into RoboRealm. With it I can do very sophisticated analysis, thinking about the problem rather than laboring over the programming details. Taking advantage of that sophistication takes time, but learning about it is a good use of my time.
|
|
|
|
John Davis [71 posts] |
9 year
|
Uploads did not succeed because .rbt was mistakenly identified as some kind of image file. I'll try again with .zip. But first, here is the top level Lego graphical program.
|
|
|
|
John Davis [71 posts] |
9 year
|
Here is the .zip file which should have everything needed to run on NXT 2.0.
 CaptureBall.zip
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
What is the exit condition for that loop? (sorry don't have lego environment installed) Does it repeat for X number of times? Or does it just repeat infinitely?
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
|
Just once. The #1 on the JPEGdiagram is hard to read.
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
And can I assume that the contest starts with someone pressing the Run program button on the NXT? Or is some other method used to start the contest?
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
|
The NXT will be powered on, but not running any of its programs. I plan to have RR running before my turn; to start the turn, I will flip a physical toggle switch to allow power to the robot drive motors. (I use remote desktop now, but the contest rules will not allow any wireless connection.) RR will decide when to name and run a program stored on the NXT.
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
Hmm, that might be a bit tricky. Since things will have to be running before you flip that power switch all the timing will be off unless there is some knowledge in the system that things have started. Can you use one of the NXT sensors like the micro switch as a 'start button'? If that's allowed, then when the system sees that switch trigger it can activate the arm movement with all the timing in the correct way. Is that possible to use?
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
Nah, not at all tricky. The NXT is powered on but doesn't do any timing until it actually is told what program to run; then all the timings are relative. The power switch I referred to was the one from the separate 7.2 V battery which furnishes servo and drive motor power. The PC and the NXT each have their own dedicated batteries. The Pololu and ESC boards get their controller power from the USB connection to the PC.
The way I hoped to use the Lego_NXT module is to run an NXT program named in variable program_to_run only in the appropriate state. I simulated that with an <If_statement module. That worked in the pipeline for one NXT program, but not for two or more.
It did work in the VBS script you provided. That script seems to use explicit estimated time delays to wait for the NXT programs to complete. If my guess is correct, this confirms my suspicion that RR cannot actually tell when the NXT program exits. It would not be my first choice but a VB or Python script can be a state machine.
Maybe the NXT isn't the best choice. I have Arduino, MSP430, and PicAxe controllers that I could use if a basic RR interface module existed. But the Lego mechanical construction is so easy...
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
If the PC is to tell the NXT what program to run, what tells the PC to start? Sorry, just trying to understand how everything starts up since that's important to how the script operates.
Correct, the NXT does NOT inform the PC when a program has completed. BUT if you are running your own programs on the NXT and can modify them you can send a mailbox message that would be received by the PC to let it know that the NXT has completed. Thus you would trigger off that in order to move to the next step.
But assuming you are running custom programs on the NXT why not just execute the entire arm movement within the NXT? Then the state machine in VB gets very simple ... or is there something within that app that the PC needs to do like look for an object?
Perhaps if you try from the other way around, i.e. have the NXT drive the PC state things might be easier. For example, the NXT would start, execute its program, when that is done it sets a mailbox code that the PC is constantly monitoring. When the PC sees that mailbox change it does what it needs to and sets the mailbox to another number that the NXT was waiting on. Sort of like a primitive handshake between the NXT and PC?
We do have an Arduino module and also a serial module that would be used with the other chips ... but I'm not sure if you'd eliminate any issues rather than replace them for others. I think the NXT should work with the right strategy.
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
STeven wrote:
Correct, the NXT does NOT inform the PC when a program has completed. BUT if you are running your own programs on the NXT and can modify them you can send a mailbox message that would be received by the PC to let it know that the NXT has completed. Thus you would trigger off that in order to move to the next step.
Thanks for the confirmation. I probably will have to use RobotC on the NXT to send mailbox messages.
STeven also wrote:
But assuming you are running custom programs on the NXT why not just execute the entire arm movement within the NXT? Then the state machine in VB gets very simple ... or is there something within that app that the PC needs to do like look for an object?
The "entire arm movement" seems to divide naturally into two separate programs: one to pick up the ball when the robot has reached it with RR, and one to drop it after the robot has turned around and gone back to the box using RR. (The arm has one servo joint which rotates the fixed length beam in a vertical plane. the other servo opens and closes the claw. The arm base cannot rotate.)
STeven helpfully suggested:
Perhaps if you try from the other way around, i.e. have the NXT drive the PC state things might be easier. For example, the NXT would start, execute its program, when that is done it sets a mailbox code that the PC is constantly monitoring. When the PC sees that mailbox change it does what it needs to and sets the mailbox to another number that the NXT was waiting on. Sort of like a primitive handshake between the NXT and PC?
I can see how that might go; I'll give it further thought after I learn how to generate mailbox messages from the NXT.
Thanks for looking at this. One additional request: can we have a preview option for replies like the one for original postings? (For the HTML-challenged)
John
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
There is a Gblock that does mail messages ... if you look at Example 1 in
http://www.roborealm.com/help/Lego_NXT.php
you will see that in action.
We'll get to work on a preview mode for the posts!
Cheers,
STeven.
|
|
|
|
John Davis [71 posts] |
9 year
|
Thanks for the advice. Just a note on progress: your simple Lego_NXT example 2 has uncovered a design flaw in the on-board NXT software.
Your Example 2 has an unused Send block at the beginning of the NXT loop. ("Unused" because the example RR module does not try to read the corresponding mailbox.) The example runs correctly for a while; you can change the RR Set Variable and see the new value on the NXT display. However, if you run for more than a minute or so, the NXT locks up and can be unlocked only by removing and reinserting the battery. I believe this is caused by the Send mailbox overflowing.
I deleted the unused NXT Send block and reran Example 2 last night; it is still working and responding to changed variables this morning.
The other thing I noticed was that, if there is any other module anywhere in the simple pipeline, then the NXT display shows only zero, not the value of the received Set variable. Maybe this is related to why Example 1 isn't working for me yet. (BTW, I am programming the NXT with version 2.0 of NXT_G and the latest NXT firmware.)
|
|
|
|
John Davis [71 posts] |
9 year
|
|
Found a workaround (fix?) for the disappearing Set variable: in the NXT_G program, feed the output from the receiving mailbox into a local variable and pass that value through to the display. Don't know if this is the right or only way, but it works.
|
|
|
|
John Davis [71 posts] |
9 year
|
More progress: mailbox sending from NXT to RR works, but only for text; numbers are not read by RR unless converted to text on the NXT first. Similar problem with logical but I haven't really looked at it.
This will make for some really awkward RR programming, but it can be done...
However, a little more testing yielded results. The number used for testing was 12345. Sent by the NXT as text, it was received by RR with the Lego_NXT module expecting text, and it was displayed as 12345. So far, so good.
If RR Lego_NXT expected a number, it displayed 875770417 (decimal) which is 34333231 (hex). This was still sent as text by the NXT and is the ASCII for "1234". This also is what we might expect. RR is working correctly.
But when the NXT is changed to send 12345 as a pure number, and RR Lego_NXT expects a number, the same RR program as in the previous program actually displays 0. My conclusion is that this is a problem with the NXT software, not RR.
(The NXT and RR programs are attached to this post. The RR is very simple, just two modules:
Display_Variables followed by Lego_NXT. The most reliable way for me to get consistent results is to exit RR and stop the NXT each time I switch programs.)
So, STeve, if you can verify this, you should update the LEGO_NXT documentation. If I am wrong, feel free to make fun of me. :)
NXT_Send_Num.zip
 Recieve_NXT_Num.robo
program.robo
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
Just a quick update, we're looking into this issue. Its a bit strange since we did verify this back when the module was first created. I suspect that something has changed in the NXT firmware or the compilation of the program. We've verified that we are seeing the issues that you mention when clearly the module was expecting back a 4 byte number. We still get a 4 byte number but the orientation of the bits seems to have completely changed and they simply don't make sense anymore. The right number is within the results (according to bit pattern matching) but the location of those bits appears to be somewhat random.
I'm experimenting with trying to upgrade the firmware on my NXT but may have bricked the thing! More on this later as we continue our investigation ....
STeven.
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
John,
Ok, we have a better handle on the issue. There is an issue with the NXT Send Message block in how it handles numbers. It has changed from 1.0 to 2.0 Apparently 1.0 worked only with integers whereas 2.0 works with floating point. The Send Message changes its format depending on the version. If you (like I did) use the G-block programs that are downloadable from our NXT page at
http://www.roborealm.com/help/Lego_NXT.php
under the examples you will see a black and white exclamation mark on the Send Message block when loaded in the most recent NXT editor. This indicates the change from 1.0 to 2.0. Our files worked as expected but when dragging in a new Send Message block that's when things got out of wack since the format switches to floating point.
We've updated the module to auto-detect this format change and process accordingly to support both 1.0 and 2.0 number formats.
What's strange is that the Receive Message block didn't have any issues that I noticed.
Latest version has this fix.
Hope this helps.
STeven.
PS. I must say that the bluetooth stability is still a huge issue with this platform! It takes a lot of patience to keep the bluetooth connection working. I hope the EV3 improves this! :-(
|
|
|
|
John Davis [71 posts] |
9 year
|
Thanks, STeven. Yes, Bluetooth is now very reliable on EV3. Our FIRST Lego League kids are now able to use it. With the NXT, it was impossible.
|
|
|
|
John Davis [71 posts] |
9 year
|
|
The latest RR (2.67.7) broke NXT 2.0 sending anything back to RR. Verified with 2.63.12 on laptop, where text at least can be sent to RR. However, the new version of RR now sends both numbers and text correctly to NXT, as tested with a NXT 2.0 switch block.
|
|
|
|
John Davis [71 posts] |
9 year
|
|
Oops! I misread something. Nothing is working correctly on my laptop either. I will have to examine my NXT program with a microscope. It is so easy to miss tiny details like wires with graphical programs like NXT_G. :-(
|
|
|
|
John Davis [71 posts] |
9 year
|
I've spent quite a bit of time on the overall project since my last post. NXT by itself works flawlessly. RR crashes frequently, quite often locking up the PC so tight that I can't bring up the Task Manager and have to hold the power button for 5 seconds to shut it down.
However, RR does find the target and drive right up to it, stopping where the arm could pick it up, if only the NXT would get the message.
I've given up on reading mailbox replies from NXT; I previously had messages writing to NXT working, letting RR wait just long enough for the action to complete. Now the simplest example doesn't work. (Bluetooth is not involved; USB direct connection always.) I could try earlier versions of RR, but I'm tired.
The NXT-RR interaction is simply too hard to debug when the NXT is programmed with LabView. I can tell the NXT never sees the number sent to its mailbox, but there is no way to tell why. Maybe BrickCC would work better, but I don't feel like working on it any more. NXT is obsolete and STeve shouldn't waste any more time on it either.
I should get a simple 2-DOF arm and control the servos directly via the Pololu Micro-Maestro. It already controls the steering servo and the ESC for the drive motors.
|
|
This Thread
