So, you can run firmware(MicroPython Project) and software(Python Project) concurrently(same time).

Question

So, you can run firmware(MicroPython Project) and software(Python Project) concurrently(same time) on an XBee3 network? Like if I turn on an xbee that has micropython uploaded to the device (firmware) that runs automatically when powered on, and at the same time I run my main Python program that has all the GUI buttons and functionality already developed... is that fine/good/normal/ok?

Overview:

I have an XBee3 network where my GUI developed in Python allows you to press buttons and stuff (widgets) which run functions that turn on/off pins on the XBee3 chip. I ran out of I/O so I attached a slave device MCP23017 i2c I/O expander on an i2c bus facilitated by each XBee remote router module acting as the master. (Each router module is the master to its respective io expander slave on its own i2c bus)

So the i2c interface on the XBee3 is accessible via MicroPython. Separately, once an XBee3 module is powered on, MicroPython runs on the chip in a way that sends the coordinator an AT command telling the coordinator that router module's MAC address so then the coordinator can reply back with a configuration profile that the router must apply to itself (the powered on router reconfigures depending on where it is plugged in).

All this works as long as:
1.) Python program and MicroPython program both may run same time.
and
2.) sending and receiving back and forth between micropython and python is fine and good.

Anything critically flawed with my understanding here?

thank you very much.

thanks!

Answer 1

Hi,

Absolutely, that is fine.

You really can think of MicroPython running on an XBee 3 device completely separately from any programs you run on a host device (such as Python in your case). The XBee 3 device is completely agnostic to what kind of program is communicating with it.

You could do any of the following:
- Have no MicroPython program on any nodes of the network, and do all the programming on one or more hosts. (This is the only option when using just older XBee models that do not support MicroPython programmability.)
- Have no programs talking to the XBee 3 at all, and run only MicroPython on nodes in your network. (That wouldn't be a very useful network without a way to get data/messages out to some host, but nothing's stopping you.)
- Run MicroPython on one or more XBee 3 nodes, and use Python/Java/C/C++/any programming language (or manual entry of API frames if you're desperate!) to manage communicating with the XBee 3. You can use User Data Relay API frames to communicate between MicroPython and the serial port.

Hope this helps.

Comments

this is very helpful. thanks!

2 further questions about this:

1.) But if you have a micropython program flashed onto a node(s), any node running micropython MUST be in REPL mode (AP=4) to run said MicroPython, no??? And if that is correct then.. yeah I've had a tough time finding good enough documentation that discusses this issue.. like if you are in REPL mode... then I cant run my python program because the nodes need to be in API mode.. I think.. and then switching between REPL and API mode would stop your flashed MicroPython from running?.. and there is no way to, in middle of program, restart the micropython without powercycling or whatever..?
 don't the modules in the network have to be in API mode (AP=1) for a network to be um... like a network?

2.) I am using an i2c io expander slave attached to the xbee(master). So, MUST I use MicroPython for this i2c or should a python i2c library type thing work fine instead?
 
This would require like being in REPL mode for the micropython to work? but then in API mode for the network to work? Apologies for my confused understanding here.

I guess in short, after you flash/upload your firmware (MicroPython program)… does it if you set the device to API mode (AP=1) and enable auto-run at start-up (PS=1).. does the firmware still run and does it stay in API mode? (like does setting PS=1 do anything funky in the background and switch API mode to REPL mode?.. lol rambling! Does the flashed micropython run if in API mode or does the device need to be in REPL mode all the time?


Helping me straighten out these confusions would be much much appreciated. thanks!

---

Hi,

If PS=1 then when the XBee turns on, the MicroPython program will automatically start and begin running "in the background". The XBee can be in API mode, transparent, or REPL mode; the program will run either way. One thing to watch out for is any unintentional calls to functions like input() or sys.stdin.read which can block, since those will only receive input when the XBee is in REPL mode.

For an I2C I/O expander like you describe, in order to communicate with it you will need to use MicroPython. I am not aware of any standard-Python libraries that would be compatible - in order to use I2C on an XBee you will do `from machine import I2C` etc.

Since Python and MicroPython are mostly compatible languages, you COULD find or write a library that is compatible with both XBee 3 (or other MicroPython devices) and Python. But in reality it's best to just stick with finding or writing a MicroPython library or program for that.

Hopefully that all makes sense.

---

ok. interesting. thank you kindly. do you mind checking my code here.

ok... so
I upload the firmware code in ATAP=4, PS=1. Consistently, when I press reset on the device... it auto-runs, sends to coordinator into a running Python program where there is a data reception callback function waiting for this message from the device with uploaded micropython.  I see what I expect. it works!

then I go in xctu, switch the device to ATAP=1 (API mode), PS still = 1. then I press reset on the device. nothing.. I don't any confirmation that the Python callback function received the message.

then I go back into xctu, set back to REPL (ATAP=4). press reset on the device. works great again (I see what I want in the python print outs and the NI gets reassigned).

here is the code (micropython sample import.. almost no changes) again, in REPL mode (ATAP=4) it clearly does what I want, then doesn't cooperate once I switch ATAP=1.. then back to working when I set ATAP back to 4.

Do you mind checking my micropython code to see if there are any obvious things im doing that would cause the blocking issue or something like you mentioned that would make this fail to successfully auto-run in API mode vs successful auto-run in REPL mode? thanks!

code:

----
import sys
import xbee
import time


# TODO: replace with the node identifier of your target device.
TARGET_NODE_ID = "COORD"
filler = "don't mind me"

while xbee.atcmd("AI") != 0:
    time.sleep(0.1)


def find_device(node_id):
    """
    Finds the XBee device with the given node identifier in the network and
    returns it.

    :param node_id: Node identifier of the XBee device to find.

    :return: The XBee device with the given node identifier or ``None`` if it
        could not be found.
    """
    for dev in xbee.discover():
        if dev['node_id'] == node_id:
            return dev
    return None


# Find the device with the configure node identifier.
device = find_device(TARGET_NODE_ID)
if not device:
    print("Could not find the device with node identifier '%s'" % TARGET_NODE_ID)
    sys.exit(-1)

addr16 = device['sender_nwk']
addr64 = device['sender_eui64']

print("Sending data to %s" % TARGET_NODE_ID)
# micropython.mem_info([verbose])... SUPPOSE TO PRINT OUT how much memory is being used or something?
try:
    # Some protocols do not have 16-bit address. In those cases, use the 64-bit one.
    xbee.transmit(addr16 if addr16 else addr64, filler)
    print("Data sent successfully")
except Exception as e:
    print("Transmit failure: %s" % str(e))

----

thanks again!

---

Hi edunn106,

Can you confirm what XBee 3 firmware you are using (Zigbee, 802.15.4, DigiMesh) and which firmware version (ATVR)?

It seems you may have found a bug with xbee.discover and AP=1. I have reported your findings internally.

As a workaround, I think you can just send a message to address 0x0 (the coordinator), assuming your goal is to find the network coordinator. This would not require a discovery operation.

---

How do you know/think this is a bug?

Do you work for digi?

I am using ZigBee, all devices are using 100A firmware version.


Lastly, no im not really trying to find the coordinator. The coordinator will already be plugged in and a python program would be running that is waiting to receive a message. The message it is waiting for is what the code above is intended for. Each router would have that code uploaded. If you grab any router module at random and plug it in, upon start-up the code I posted above would run and the router would send a message containing it's MAC address to the coordinator. Then the Python program would receive that message through a data reception callback function, take the MAC address and assign that MAC's device a new Node Identifier (NI) depending on some stuff.

The work around I am presently working on is as follows:

I am setting Join Notification (JN) to enabled (1) on each router. so when you plug in a router it sends an API packet to all the network nodes. Right now im trying to figure out how to catch this message and extract the MAC address out of that message, which is a series of bytes. If this works then that's great.. but I would still need to use the code above to send i2c bus device information that might be attached to the xbee routers.

---

Yes I am a Digi employee.

At this point I think you should contact Digi Technical Support at tech.support@digi.com for more assistance.

---

ok will do.

how do you think/know it is a bug? just because it should work?

also why the redirection to digi support? do you mean just for more general assistance or specifically about this bug / about this specific matter of questions?.. like what should I ask  tech.support@digi.com that's different than this forum thread?

also thank you very much for your assistance.

---

Yes, I would expect it to work, but from your experience and a quick look at the firmware code it appears it does not.

This support forum is more or less community driven (some of the community being Digi employees). Tech support should be able to assist you more directly with this and other specific questions.

You are welcome.

---

the "from my experience part" is actually highly probable that I am doing something fundamentally wrong lol. but ok cool.

If the tech support line is more helpful/available than this forum then I will be super in luck.

thanks

---

One more thing if you're willing..

do you know, in Python, how to get a specific segment of a received Join Notification frame?

I know its in here somewheres:

https://xbplib.readthedocs.io/en/latest/api/digi.xbee.reader.html?highlight=Join%20notification

In consoles working mode with the serial connectioned opened between router and coordinator I see the many bytes that were sent from me plugging in a router module with JN=1 set. just need to grab it in python and extract the MAC address... data reception callback function?.. some magical phrase I should whisper?

---

Yes, you can use add_packet_received_callback to get told about every frame output by the XBee. You can then get the information from there. https://xbplib.readthedocs.io/en/latest/api/digi.xbee.reader.html?highlight=Join%20notification#digi.xbee.reader.PacketListener.add_packet_received_callback

Any questions specifically about the xbee-python library can also be asked on the library's GitHub: https://github.com/digidotcom/xbee-python. That way the developers in charge of the library can respond. You can also search past issues there in case a question has already been asked.

---

awesome, I got that to work. At least primitively. I just print out this ugly cluster of the packet that it sends from the join notification. its a dictionary of crud. the bytearray is sent in right to left bit index format.

for this I just had it print out whatever packet it sent through the callback function parameter.

do you happen to know of any "prettier / easier" way to print out or work with the packet that it sends? instead of this raw dictionary thing?

---

hello,

question about that add_packet_received_callback:

if a packet is received does it trigger that functrion no matter what? im trying to make it trigger on the first packet (the one I am interested) but instead it triggers there and then now any time I do anything that requires a packet sent in the background, it gets triggered and the callback does its thing. I don't want that.

---

Yes it triggers no matter what. If you want to disable the callback you should be able to use del_packet_received_callback ( https://xbplib.readthedocs.io/en/latest/api/digi.xbee.reader.html?highlight=Join%20notification#digi.xbee.reader.PacketListener.del_packet_received_callback ). Otherwise you could write your code to ignore calls/return immediately if you don't care about the packet.

---

yeah, that's what im trying to implement.

I tried to basically have this flow but doesn't quite work out yet:
------------
local_device.add_packet_received_callback(my_packet_received_callback1)
------------
which goes to:
------------
def my_packet_received_callback1(xbee_packet):
    local_device.del_packet_received_callback(my_packet_received_callback1)
    my_packet_received_callback(xbee_packet)
--------
which, I thought, should delete the callback and then send you to:
--------
def my_packet_received_callback(xbee_packet):
--------
which finally manipulate the received packet... and then recalls the function that the initial callback_add trigger while loop is in, thereby awaiting another join notification packet (which is my intention after im done manipulating the first reception event).


does that make sense to you? because my issue was that I would receive the packet through the callback trigger, then hops me to the function, but then when I set other configs for the an xbee of course that would prompt packets sends that would trigger the callback again...

so through the above code I was trying to hide/delete the callback trigger so I could just get the first packet and then manipulate it and then when done recall the function that reimplements the callback trigger.
but its not quite right.

at first I was deleting the callback right after the callback_add In the while loop but of course the callback_add would send you to the callback function through the passed parameter...

so then I tried to delete the callback in the function that the callback_add passed parameter would send you to... but then that's weird cuz the callback_del has the same function passed as the parameter lol.. so hence the code flow I posted above:
 callback_add -> callback function that deletes then passes on the xbee_packet to the next function that manipulates the xbee_packet

any ummm obvious terribleness?

thanks

---

I think you should submit that as a question/issue on the library's GitHub page ( https://github.com/digidotcom/xbee-python/issues ) so the library developers can assist you.

---

done.

thanks

---

It looks like xbee.discover() on XBee3 Zigbee doesn't work when ATAP != 4.  That's a bug that isn't present in the 802.15.4 and DigiMesh firmware.  It's also possible that xbee.receive() won't work either.  There are firmware issues with selecting where to deliver the frames.

---

that reaffirms what tckr said in reply to my question. Bummer, would be nice to use. That bug may alter my component selection here soon.

Was going to use an i2c slave device with the xbee as a master to facilitate more I/O ports... but since that bug exists and hence I won't be able to request/transmit the I/O port data from the i2c slave device... I might have to go with ditching the i2c slave device cheaper solution and instead go with just doubling the amount of xbee router devices I will be using in order to facilitate more I/O ports without using micropython.

This bug won't be fixed anytime soon for ZigBee mesh devices im guessing, yeah?

---

Is there a reason you can't continue running with ATAP=4?  If you don't have anything connected to the serial port of these routers, then it shouldn't matter what ATAP setting you're using.  If you need to send data out that serial port, you can just have MicroPython print() statements pass it out.

---

mmmm… yes there is a reason.

which is..

I need all the devices in API mode to establish a network right?

My concise system overview is as follows:

network of ~ 20 XBee3 modules, coordinator, else routers...

PC running main Python program that holds the GUI stuff and defines most of the functionality, xbee onboard pin manipulation stuff... then at first I was going to have micropython flashed on each router module in which, at start up each device would send a message at start-up containing its MAC to the coordinator which would be processed in the Python program and then the router would be assigned a new NI through that MAC address as an identifier(I've almost successfully bypassed this due to the firmware bug issue by just having the JN=1 for each router and would do all that reassignment through that API frame parsing in python instead of using start-up micropython transmit functionality... after that start-up subprocess is complete for the whole network, there would be more micropython that would essentially be waiting to have i2c pin status requested and sent into the python program from router to coordinator.

back to your question... so my current understanding (which at any point could have major fundamental holes) is that yes, the way I have the system implemented currently, all the modules would need to be in API mode because.. well in API that is how you have a multi-node network established right? in ATAP=4 mode you can't like open the coordinator device and then discover router devices into the network.. you cant establish a multi-node network with ATAP=4 mode right? Like when I run my python program that basically opens the coordinator and then simply discovers the router modules via their NI string.. if not in API mode the program just immediately says "hey you're not in API mode so don't come back until you are."


but yes. for multi-node network establishment you need API mode... not serial point-to-point mode (ATAP=1 or 4)

unless I fundamentally rework the flow of my design... which im trying to avoid after a couple months of work lol.

---

The nodes on a network can have any combination of ATAP settings.  The coordinator can use ATAP=1 and the routers can use ATAP=4.

ATAP is only used to control how the serial port of the XBee works.  0="transparent" mode (raw, unframed data), 1 and 2 are API modes (framed data) and 4 is MicroPython (REPL and running program output on serial port).

---

ok I lied, when I switch the routers to ATAP=4 and then run the python where it discovers them like usual, program runs fine..

what the fudge. the intro documentation makes you think API is for multi-node.. AT is for 2 devices in total.

guess not.

---

ok so..

in my network of 3 devices (1 coord, 2 routers)

if both routers are in AT modes Coordinator in API... python program ive developed works fine.

if all 3 in AT mode... then I get that immediate error saying Unsupported operating mode: AT mode


So the coordinator is what matters with the whole api vs at modes??? only the coordinator lol?

so I can indeed have the coordinator in API mode..

and all the routers in AT=4 or AT=1 mode and poof all the things ive been going on about the past week don't matter at all?

I can send all the stuff I want in python or micropython, transmit receive between the two no problem??

whats the point of having the routers in api vs at modes then.. why aren't routers/end devices just always in AT modes without the option to switch if they can do it all in AT modes?..

but then I guess the firmware transmit ******* I did yesterday/today with having the JN=1 on each router device so it sends an API packet wont work if the routers are in AT mode.. cuz then they won't send the API packet that I've parsed...

so I can just go back to my micropython start-up thing maybe


thanks btw

---

It's much simpler than that.  Re-read my message about what ATAP means.

AT mode (0) means the serial port is configured for transparent data.
API mode (1, 2) means the serial port is configured for framed data.
MicroPython mode (4) means the serial port is configured for the MicroPython REPL and running program.

Different customers will use different modes for their deployed network.  In general, if you have another microprocessor interfaced to the XBee via its serial port, you'll want to run in "API mode" so you're sending and receiving API frames.  If you have a "dumb" device that can't parse data, you typically run in "AT mode" to transparently send/receive serial data.

It has nothing to do with whether it's a coordinator or a router.  It has everything to do with what you're going to connect to the serial port.  Your coordinator has a PC running Python code expecting an XBee module in API mode.  Your routers don't (as far as I can tell) have ANYTHING wired to the serial port.  So your routers can use ATAP=4 mode to work around the current XBee3 Zigbee bug where xbee.discover() doesn't work otherwise.

---

BTW, please send a link to the introductory documentation that wasn't clear, and we'll try to improve on it.