Using Lvgl in color radio widgets.

[philmoz]

I have made a start on being able to use Lvgl for the UI in widgets (obviously color radios only).

This requires a new API to define the UI layout, and is incompatible with the existing direct drawing API - a widget can use one or the other; but not both.

For example, this script:

local options = {
  { "Timer", TIMER, 1 },
  { "Color", COLOR, COLOR_THEME_SECONDARY1 },
}

local function create(zone, options)
  local widget = { zone=zone, options=options, counter=0 }
  return widget
end

local function update(widget, options)
  widget.options = options
end

local function background(widget)
end

local function layout(widget)
  lvgl.clear()

  widget.ctrLbl = lvgl.label("CTR", 0, 0, 0, 0, widget.options.Color, MIDSIZE)
  widget.ctr = lvgl.label("", 80, 0, 0, 0, widget.options.Color, MIDSIZE)

  widget.tmrLbl = lvgl.label("TMR", 0, 50, 0, 0, widget.options.Color, MIDSIZE)
  widget.tmr = lvgl.label("", 80, 50, 0, 0, widget.options.Color, MIDSIZE)

  widget.box1 = lvgl.rectangle(50, 80, 30, 30, 4, COLOR_THEME_ACTIVE)
  widget.box2 = lvgl.filledRectangle(90, 80, 30, 30, COLOR_THEME_EDIT)
  widget.cir1 = lvgl.circle(50, 120, 30, 4, COLOR_THEME_ACTIVE)
  widget.cir2 = lvgl.filledCircle(90, 120, 30, COLOR_THEME_EDIT)

  widget.arc = lvgl.arc(0, 80, 15, 4, 0, 210, COLOR_THEME_WARNING)
end

local function refresh(widget)
  widget.counter = widget.counter + 1

  lvgl.setText(widget.ctr, widget.counter)
  lvgl.setText(widget.tmr, model.getTimer(widget.options.Timer).value)

  lvgl.setPos(widget.ctr, 80 + widget.counter % 20, widget.counter % 20)
  lvgl.setSize(widget.box2, 20 + widget.counter % 20, 20 + widget.counter % 20)
end

return { name="Lvgl Test", options=options, create=create, update=update, refresh=refresh, background=background, layout=layout }

Produces this output:

Instead of redrawing the widget every time refresh is called, the layout is done once (or when the size changes), then the refresh function only needs to update the attributes of the Lvgl objects.

I would like to get feedback on what the new API should look like - so any thoughts here would be appreciated.

At the moment interactive elements are not supported (i.e. buttons, text boxes, etc); but should be possible in the future.

[stronnag]

Please ensure the old API remains supported "as is". I do not have either the time or energy to migrate a complex widget (i.e. the INAV Lua to a new API), and I doubt I'll easily find a new maintainer to do so.

I don't mean to discourage you from this endeavour, which I'm sure will be appreciated by many. However, for the reluctant maintainer of a legacy OTX/ETX widget, backwards compatibility is everything.

[gagarinlg]

The current plan is to slowly add the new API over the next releases and then have at least one release with both APIs. How we will maintain two API remains to be seen.
@philmoz great work!

[frankiearzu]

@philmoz good job! As others said, keep the Lcd API the same.

I guess the idea is to get this closer to to how the /WIDGETS/LuiGui works.
I know that im probably getting ahead of the POC, but besides the drawing functions, I think more higher level components would be beneficial for LUA by providing similar capabilities to native EdgeTX. for example, Forms, checkboxes, dropdown lists, editNumber, editText, sliders. That will allow LUA not to be different of how we do use inputs, and save a lot of LUA code.

Some things not necessary needs to be only color.. if we can get access to similar components in B&W radios, the code will look similar from LUA point of view regardless if it is B&W or color.

[pfeerick]

This is "just" exposing access to the LVGL controls, which in turn should make Lua scripts a lot more responsive and powerful, since the individual controls can be refreshed, rather than the whole screen/widget zone redrawn whenever there are any changes. LVGL isn't used on B&W at all so isn't really an option, and given size constraints, may not be an option at all.

[offer-shmuely]

great POC, looks promising, it may give a boost to the color screens apps (happy to see that we are talking only on color screens!)
this I assume the low-level functions, that will later will be the base for the buttons/tables/....
can we see a text that the font is growing slowly like the rectangle (this is the most important one)

also, the "layout=layout" looks to me very limited
it assume only one layout during the life of the widget.
but I like to replace layouts, during the lifetime of the widget.
can we do it differently? like lvgl.setLayout(myLayout2)

[philmoz]

also, the "layout=layout" looks to me very limited it assume only one layout during the life of the widget. but I like to replace layouts, during the lifetime of the widget. can we do it differently? like lvgl.setLayout(myLayout2)

Currently the layout function serves two purposes - first it lets the firmware code know the widget wants to use the Lvgl UI. This turns off the automatic erase of the underlying screen area. Secondly it is called whenever the widget size changes so you can create or modify the Lvgl objects used to define your widget.

How you setup the Lvgl objects to create your actual UI is up to you. If you want multiple layouts that should be possible - the API will allow you to create frames (empty Lvgl containers) that can then contain other UI objects. Individual objects (including frames) can be shown or hidden as needed.

I may end up removing the 'layout' function as the existing 'update' function could serve the same purpose. In this case the 'create' function would then need to set some flag to turn on Lvgl mode.

I hope to be able to expose the styled objects used in the main UI - buttons, toggle switches, sliders etc. However I need to figure out how the interaction with these elements is handled. I.E. when you tap a button how does this get to the widget script code.

[CapnBry]

Pleased to see that someone is thinking about how to get away from raw drawing UI!

• +1 to what offer-shmuely said about layouts needing to be able to change on the fly in response to user actions and not just once.
• The color also needs to be able to change in refresh as well. I don't think your example was meant to be exhaustive but just want to make sure color change is included.
• There's definitely a performance benefit to drawing this way but apart from standard themed controls, with a RAM penalty for each control. The problem is doubled because you still need code to update it every refresh, and also to track the IDs of the sources that the controls are representing, previous values, etc just like existing widgets do now. This could be improved by allowing binding of controls to EdgeTX sources and getting an optional observer callback to format the value / color it etc. Then you have no code in your refresh body since all the updates are just called as needed. I feel like that's where the greatest benefit is with any lua changes, since just rewriting all my code to use lvgl.setText instead of lcd.drawText doesn't add much beyond the render speed benefits. Data binding also adds a speed benefit in that you're not even updating every dynamic control every frame-- only the elements that change are redrawn.

[JimB40]

Good direction Phil.
Beyond what was said before, main oppotrunity I can see is using lvgl objects to create standarized and full-featured widget's options (configuration). Now we have to write own LUA module to get input, validate and store it. Not mentioning that getting for example text as option in plain PIA.
Using LVGL ojects instantly we can get acces to checkboxes, radiobuttons, proprer text input or validated numeric inputs (min,max, etc).
If this evolve into direction of creating standarised forms (lines) with label and LVGV control - UI for LUA like CRSF protocol (TBS or ELRS) configurator or Spektrum Forward Programing will be piece of cake.

[philmoz]

This could be improved by allowing binding of controls to EdgeTX sources and getting an optional observer callback to format the value / color it etc.

While this might simplify the Lua script code I suspect it could have performance issues. The data structure do not allow direct binding to radio or model variables (due to bitfields being used). So binding must go though getter/setter functions.

This would mean that, instead of a single call from the C code to the Lua refresh function, each object would need to make one or two such calls.

[offer-shmuely]

Indeed I believe as well that it will smarter to getter/setter from the lua to the firmware, as it simply complex implementation and lockups

[offer-shmuely]

can we see a text that the font is growing slowly like the rectangle?
From small to big fonts?

[gagarinlg]

That is not possible. We have a fixed set of font sizes, the same that you can access via Lua. The fonts are not rendered at runtime, but pre-rendered to specific sizes.

[offer-shmuely]

Hmmmmm,
I thought the lvgl already use true type fonts 😥

[CapnBry]

While this might simplify the Lua script code I suspect it could have performance issues. The data structure do not allow direct binding to radio or model variables (due to bitfields being used). So binding must go though getter/setter functions.

Oh yeah I for sure know it isn't easy to implement in EdgeTX because the infrastructure isn't there but a thin wrapper around lvgl doesn't change much from the lua developer standpoint. Instead of just drawing text in one line (current lua) you'd just be changing it to become multiple lines (one to create the control and then set attributes, then one to update the text).

I know that this draws faster but is that the only goal of this? If someone is going to add an entire new system to the lua, I feel like there's a great opportunity to not have to manage all the same stuff an employ a more verbose syntax to get those values on the screen but perform better. Binding a control to a source eliminates all the code of updating the text, caching the ID to get the value faster, comparing the previous value to see if it needs to be updated, etc. Maybe I'm just a die hard event-driven programmer though haha.

I like what you've got so far (although I'm unsure about the 4 numbers in the middle which will likely have me pulling up the documentation every time to remember what order the 4 values are in) but let's blue sky a little before we start discarding ideas just because the current codebase doesn't already support it 😀 It doesn't support this either, but you wrote it in no time at all so now it does!

[philmoz]

although I'm unsure about the 4 numbers in the middle which will likely have me pulling up the documentation every time to remember what order the 4 values are in

Would using a table for the parameters be preferred:

    lvgl.label({text="CTR", x=0, y=0, w=0, h=0, color=widget.options.Color, font=MIDSIZE})

[offer-shmuely]

Looks unusual
But maybe you can do the table just to the positioning

lvgl.label("CTR",
{x=0, y=0, w=0, h=0},
widget.options.Color,
MIDSIZE)

Why do you have w&h for label?

Btw, on my script i replaces the MIDSIZE to FONT_6 FONT_8 FONT_12

[philmoz]

I'd prefer either individual variables or a table, not a combination.

With a table optional values can be left out - e.g. width and height for a table. If not set (or set to 0) the label will auto resize. If you want a fixed size you can specify the values.

Colors and fonts will work exactly the same as they do now - if you want to create aliases for the font names they will still work.

[CapnBry]

It certainly is clearer and I like the optionality aspect, but after thinking about it for a bit the original form might be better just strictly on its lower memory footprint. Wouldn't passing tables for every control greatly increase the size of the compiled script lua code in RAM compared to just passing numbers? I do love embedded programming where we have to think "well that seems nice but it also takes 1KB extra sooo..." 😅

[JimB40]

Is there a artifact (build) or branch with code for what you do @philmoz?

[philmoz]

Not yet, still a lot of work to do.
When 2.10 drops I hope PR #4407 will get merged soon after as the base for 2.11.
The PoC for this is based on PR 4407 so hopefully there will be something to test here soon after that.

[philmoz]

API summary moved to PR #4887

[philmoz]

A couple of other notes:

• General pattern would be to create the layout and UI elements in the widget 'update' function, save the references for any elements that will be changed and update them in the 'refresh' function. You may want to check if the actual widget size has changed before rebuilding the layout (update can also be called if other options are changed).
• When changing the value of a UI element attribute it is ok to call the attribute function even if the value has not changed. The firmware code will keep track and only change the underlaying value when necessary. No need to store and compare values in the Lua code.

[philmoz]

Assuming the CPU usage logic I added is correct then:

Direct draw BattAnalog widget - total CPU usage (mixer, widgets, scripts, etc) = 100%

Lvgl BattAnalog widget - total CPU usage = 36%

[philmoz]

Adding the FPS into the mix makes the performance improvement even more dramatic.

The Lvgl widget runs at 20 FPS (the max rate the gui task runs at) with 64% CPU to spare..
The direct draw widget can only manage 16.2 FPS while using 100% CPU.

[offer-shmuely]

awesome!
we can save even more wasted processing power, if you let the widget/script define it's needed FPS.
for the batt-check/batt-analog I can set it to FPS=4 without any effect on the user
for gauge-rotary I think a FPS=8-10 will be enough, and the layout there is match more complex!!
(as long as the background function continue to be called frequently)

[philmoz]

Interesting idea, I will keep it in mind.

I don't think an actual FPS setting would be possible; but a 'refresh every N cycles' might be doable.
If set to 1 you get 20fps (best case), 2 = 10fps, 3 = 6.6fps, etc.

[offer-shmuely]

I see it now,
Look good enough to me.

i still use the “fps=6” in the api, and calculate it internally to 3, so if some day we will have a H7 radio, with default of 60fps, the api will still hold

[philmoz]

There is a small increase in memory usage:

Compiled .luac files (from TX16S):
Direct draw widget = 16935 bytes
Lvgl widget = 18204 bytes

Max memory used when running:
Direct draw widget = 194K
Lvgl widget = 200K

[gagarinlg]

This is pretty much a don't care on the color radios

[philmoz]

For reference here is the current version of the Lvgl converted BattAnalog script.

BattAnLvgl.zip

[offer-shmuely]

to allow the functions to be more organized, in autocomplete & documentation
it may be better that thire names will have the same prefix
i.e.
instead of:

lvgl.rectangle()
lvgl.filledRectangle() 
lvgl.circle()
lvgl.filledCircle() 
lvgl.triangle()
lvgl.filledTriangle()

it is better to use:

lvgl.rectangle()  
lvgl.rectangleFilled()
lvgl.circle()  
lvgl.circleFilled()
lvgl.triangle()
lvgl.triangleFilled()

[philmoz]

I was probably going to make 'filled' a property rather than a separate type.

There is no triangle equivalent in Lvgl - how important is this?

[offer-shmuely]

That is even better.

[offer-shmuely]

I used the triangle to draw the needle of the rotary-gauge.
we can think of another way…

[philmoz]

This might work as a replacement for the complex gauge drawing. It should be possible to expose this through the api. https://docs.lvgl.io/8.0/widgets/extra/meter.html

[offer-shmuely]

What about the FPS?
Is it technically possible to let the script to control it?
It will save a lot of processing power
I believe that many widget scripts does not need most of the time more than 2-4 FPS

[philmoz]

See earlier comment.

[philmoz]

Some progress on supporting EdgeTX styled controls in a stand alone script.
There may be some additional complexity here due to the way events are being passed to the script - I will try and make it as compatible as possible.

standalone.script.mp4

[philmoz]

[philmoz]

Also supports scrolling:

[gagarinlg]

Very cool!
Now we have to hope the LUA script authers are willing to use this API in the future.

[offer-shmuely]

Well done 👍
Looks very promising!
Do you have a video of the scrolling?

[offer-shmuely]

Will the lvgl widgets be available to the lua script?
I saw there amazing ones

https://docs.lvgl.io/8.0/widgets/extra/meter.html

[philmoz]

I've added support for the Lvgl 'meter' widget.
There are some limitations - you can't control the font size or color of the scale labels.

lvgl.meters.mp4

Widget script used to generate above video:

local opts = {
}

local wgt

local function create(zone, opts)
  wgt = { zone=zone, opts=opts, counter=0, bg_ctr=0 }
  return wgt
end

local function update(wgt, opts)
  wgt.opts = opts

  lvgl.clear()

  local lyt = {
    {type="rectangle", x=0, y=0, w=120, h=120, color=WHITE, filled=true, rounded=60,
        children={
            {type="meter", x=60 , y=60, radius=60,
                scales={{
                    min=-100, max=100, angle=270, rotate=135,
                    ticks=41, tickWidth=1, tickLen=10, tickColor=COLOR_THEME_PRIMARY1,
                    majorNth=8, majorWidth=2, majorLen=15, majorColor=COLOR_THEME_PRIMARY1, labelGap=10,
                    dotSize=6, dotColor=ORANGE,
                    indicators={
                        {type="arc", w=2, color=BLUE, rmod=0, startPos=-100, endPos=-60},
                        {type="arc", w=2, color=RED, rmod=0, startPos=60, endPos=100},
                        {type="lines", startColor=BLUE, endColor=BLUE, localFade=true, wmod=0, startPos=-100, endPos=-60},
                        {type="lines", startColor=RED, endColor=RED, localFade=true, wmod=0, startPos=60, endPos=100},
                        {type="needle", w=1, rmod=-10, color=BLACK, getPos=(function() return -100 + wgt.counter % 200 end)},
                        {type="needle", w=1, rmod=-10, color=GREEN, getPos=(function() return 100 - wgt.counter % 200 end)},
                    }},
                },
            },
        }
    },
    {type="rectangle", x=120, y=0, w=120, h=120, color=WHITE, filled=true, rounded=60,
        children={
            {type="meter", x=60 , y=60, radius=60,
                scales={{
                    min=0, max=100, angle=270, rotate=90,
                    ticks=11, tickWidth=1, tickLen=15, tickColor=COLOR_THEME_PRIMARY1,
                    majorNth=2, majorWidth=1, majorLen=15, majorColor=COLOR_THEME_PRIMARY1, labelGap=15,
                    indicators={
                        {type="arc", w=5, color=RED, rmod=0, startPos=0, getEnd=(function() return wgt.counter % 100 end)},
                        {type="arc", w=5, color=GREEN, rmod=-5, startPos=0, getEnd=(function() return wgt.counter % 200 / 2 end)},
                        {type="arc", w=5, color=BLUE, rmod=-10, startPos=0, getEnd=(function() return wgt.counter % 300 / 3 end)},
                    }}
                },
            },
        }
    },
    {type="rectangle", x=240, y=0, w=120, h=120, color=WHITE, filled=true, rounded=60,
        children={
            {type="meter", x=60 , y=60, radius=60,
                scales={
                    {
                        min=0, max=60, angle=360, rotate=270,
                        ticks=61, tickWidth=1, tickLen=10, tickColor=COLOR_THEME_PRIMARY1,
                        indicators={
                            {type="needle", w=1, rmod=0, color=BLACK, getPos=(function() return wgt.counter % 60 end)},
                        }
                    },
                    {
                        min=1, max=12, angle=330, rotate=300,
                        ticks=12, tickWidth=0, tickLen=0,
                        majorNth=1, majorWidth=1, majorLen=15, majorColor=COLOR_THEME_PRIMARY1, labelGap=10,
                        dotSize=6, dotColor=BLACK,
                        indicators={
                            {type="needle", w=1, rmod=-20, color=BLACK, getPos=(function() return wgt.counter % 720 / 60 + 1 end)},
                        }
                    },
                },
            },
        }
    },
  }

  local refs = lvgl.build(lyt)
end

local function background(wgt)
  wgt.bg_ctr = wgt.bg_ctr + 1
end

local function refresh(wgt)
  wgt.counter = wgt.counter + 1
end

return { name="Lvgl Meters", options=opts, create=create, update=update, refresh=refresh, background=background, useLvgl=true }

[offer-shmuely]

looks very nice, it will be easier now to use more "meter" easily. mainly on the c++ side.
however, it keeps all the power of code on the firmware level.
in opposes to keyboard & checkbox, that we need it "as is", meter like this, will be used in different scenario, for different purpose, so it should be very flexible for new stuff, and not for a single (cool!!) widget.

We like the Lua to be free to define it's own widget variations.
is it possible to move more of the code to the lua level? to show the power of the scripting?
without using predefined [type="meter"] and [type="needle"] ?

thanks for the great work.

[philmoz]

however, it keeps all the power of code on the firmware level. in opposes to keyboard & checkbox, that we need it "as is", meter like this, will be used in different scenario, for different purpose, so it should be very flexible for new stuff, and not for a single (cool!!) widget.

We like the Lua to be free to define it's own widget variations. is it possible to move more of the code to the lua level? to show the power of the scripting? without using predefined [type="meter"] and [type="needle"] ?

All I have done is expose the properties of the Lvgl 'meter' widget which you asked for. You can create any 'meter' layout you want using all these properties as described in the Lvgl documentation for the widget.

The Lvgl 'meter' widget is a complex object and can only be built using the properties it provides. Things like the 'needle' type come from Lvgl not the EdgeTx firmware.

[offer-shmuely]

Thanks for the explanation.
I did not notice the widgets implanted by the lvgl library, i thought it was only samples.