Embedded Yocto - Python Standard Library AST Module

Question

Hi All,

I am using the Digi Embedded Yocto minimal (non-graphical) image on a ConnectCard iMX28 jumpstart board.

I have made judicious modifications to my local.conf and custom layers to add new features and a few Python packages required in my firmware application.

I now need to include support for Python's Abstract Syntax Trees (AST) module.

I expected this to be part of the "core" Python functionality included in the image as this is part of the Python Standard Library, but it appears at first glance that the ast module is not available in the DEY image.

Any insights are sincerely appreciated.

Thanks.

Answer 1

Upon further investigation, it looks like this standard library module is split out into the python-misc module.

One can infer how the features of interest are divided among packages by looking in the packages-split directory that is generated upon the completion of a Yocto build.

For my ConnectCard i.mx28 jumpstart board, the relevant directory to inspect is

~/workspace/ccardimx28js/tmp/work/armv5te-dey-linux-gnueabi/python/2.7.3-r0.3/packages-split

Grepping the files there shows that <ast> is in the python-misc package. Therefore, one can modify the local.conf to include this package in the install. In so doing, the standard library support for the ast module is recovered.

Answer 2

AST is part of the standard library of python, you don't need to install it separately. I also beleive it's pure python so you can add it manually by importing this file:
https://hg.python.org/cpython/file/2.7/Lib/ast.py

Comments

Hi Leonid.

Exactly; AST is part of the standard library, yet for the Digi Embedded Yocto distribution, it appears to be missing, and this is the source of the confusion.

If I try to import the ast module on the embedded target's Python image, I get an import error, e.g.

>> import ast
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
Import error: No module named ast

This behavior is also consistent with the fact that inspection of /usr/lib/python-2.7 on the target device does not include the ast.py file.

As you say, I can do take the steps to add the ast.py file manually or bake it into the image individually. However, it seems like this is a shortcoming of the DEY build of the standard Python library.

Thanks.

---

did you try to create /usr/lib/python-2.7/ast.py
from the source in the link above?

---

I believe that would work to bake the Python file directly into the image.  However, I was interested in the more fundamental question of why this standard library functionality is missing; in short it seems like an oversight at first that one would have to workaround such things manually.

Upon further investigation, it looks like the Yocto model carves up / splits things out into many different packages, perhaps for added flexibility to come up with lean images. The documentation on how these things are split is hard to find.