periph/gpio: new expandable GPIO API #14602

gschorcht · 2020-07-24T11:39:03Z

Contribution description

Please note: Even if this PR should be compilable with the current master, the new API provided with this PR cannot be tested standalone. At least one MCU implementation is required for testing. Follow-up PRs with two MCU implementations are PR #14610 for STM32 and PR #14607 for ATmega.

This PR provides a new GPIO API that

allows the expansion of GPIO ports using GPIO expanders, such as I2C I/O expanders,
allows consistent access to GPIO pins of the MCU and GPIO expanders with the GPIO_PIN(port,pin) macro
allows a hardware-oriented implementation of access to the pins of a GPIO port and
does not require any changes to existing applications.

It is the result of the discussion in POC/RFC PR #12877. The new GPIO API consists of a high-level API and a low-level API:

The pin-oriented high-level API is compatible with the "Legacy GPIO API" and is intended for use by the application to access single GPIO pins.
The port-oriented low-level API provides functions for port-oriented access to the GPIO pins. It has to be implemented for each GPIO hardware component (MCU GPIO ports and GPIO expanders). The port-oriented functions of the low-level API are
```
gpio_mask_t read(gpio_port_t port);
void set(gpio_port_t port, gpio_mask_t pins);
void clear(gpio_port_t port, gpio_mask_t pins);
void toggle(gpio_port_t port, gpio_mask_t pins);
void write(gpio_port_t port, gpio_mask_t values)
```
These functions are used by the high-level API to realize the pin-oriented functions. They can also be used by drivers or the application if several pins of a GPIO port are to be changed simultaneously using the definition of GPIO pin masks.

Implementation Details

GPIO pins

A GPIO pin of type gpio_t is now defined as a structure of a GPIO port of type gpio_port_t and a pin number of type gpio_pin_t. gpio_t can't be overridden anymore.

The number of pins per GPIO port is defined by the width of type gpio_mask_t which in turn is determined by module gpio_mask_8bit, gpio_mask_16bit or gpio_mask_32bit. Each driver for GPIO ports including the MCU implementation has to enable the according module. The width of type gpio_mask_t is then the maximum width of all different GPIO ports used in the system.

GPIO ports

The GPIO port of type gpio_port_t and can be either

the register address in case of a MCU GPIO port and if all MCU GPIO ports follow a consistent address scheme,
the address of a device descriptor of type gpio_dev_t in case of a GPIO expander port, or
the port number.

The port number is a sequential number that goes from 0 to GPIO_EXP_PORT-1 for MCU GPIO ports. For GPIO expanders, the port number is derived from GPIO_EXP_PORT and their index in the GPIO device table gpio_devs as offset.

In the case of GPIO expander ports, the port is defined as a reference to a device descriptor of type gpio_dev_t. This device descriptor contains

a reference to a device-specific descriptor which is used by the associated device driver and
a reference to a driver interface of type gpio_driver_t which contains the references to the low-level GPIO API functions of the associated device driver.

Device descriptors of type gpio_dev_t are stored for all existing GPIO expander ports in a device table for GPIO Ports gpio_devs, where each entry contains a device descriptor for exactly one GPIO expander port. If a GPIO expander provides multiple ports, each port has to be defined as separate device of type gpio_dev_t. The device table for GPIO expander ports gpio_devs is automatically generated from the GPIO_EXP_PORTS macro definition which has to be defined either by the board definition or by the application in a separate file gpio_exp_conf.h.

In the case of MCU GPIO ports, the port defines just the register address. The mapping to the functions of the low-level API is realized by the special driver interface instance gpio_cpu_driver which is created automatically from the MCU low-level functions gpio_cpu_*.

MCU Low-level API Implementation

The MCU implementation of the low-level API has to define

the low-level API functions with prefix gpio_cpu_*,
the width of its GPIO ports using module gpio_mask_8bit, gpio_mask_16bit or gpio_mask_32bit,
if register addresses are used as port definitions the macros
- GPIO_CPU_PORT and GPIO_CPU_PORT_NUM
- GPIO_CPU_PORT_BASE and GPIO_CPU_PORT_MASK or alternatively GPIO_CPU_PORT_IS
the feature periph_gpio_exp.

How the new GPIO API works

If the MCU implementation does not support the new GPIO API, that is the periph_gpio_exp feature is not provided by the MCU, the legacy GPIO API is used. GPIO expanders can then only be used via the device-specific driver interface.

If the MCU implementation supports the new GPIO API, that is the periph_gpio_exp feature is provided by the MCU, the GPIO API works as follows:

Default case where GPIO expanders are not used (gpio_exp module is not enabled).
In this case, high-level API functions call the MCU low-level functions gpio_cpu_* directly. There is no MCU driver interface gpio_cpu_driver and no device table for GPIO expander ports gpio_devs. So only slightly more RAM than with the legacy GPIO API is needed for the port member in the GPIO pin variables. The performance is the same as with the old GPIO API or even better, depending on the MCU low-level implementation, such as for ATmega and STM32.
GPIO expanders are used (gpio_exp module is not enabled)
The low-level API functions are called indirectly via a driver interface. For this purpose, the high-level API first determines with the macro GPIO_CPU_PORT_IS whether the specified port is an MCU port and uses either the driver interface gpio_cpu_driver to call the MCU low-level functions indirectly or the driver interface of the corresponding GPIO expander from the device table gpio_devs. In this case, on Harvard architectures such as ATmega, additional RAM is required for the MCU driver interface gpio_cpu_driver and the device table for GPIO expander ports gpio_devs. Of course the performance is lower than with the old GPIO API due to the additional checks and the indirect call of the low-level API functions. But for ATmega it is still better because of the more efficient implementation of the low-level functions.

This approach is implemented as following:

#if MODULE_GPIO_EXP
static inline const gpio_driver_t *gpio_driver_get(gpio_port_t port)
{
    if (GPIO_CPU_PORT_IS(port)) {
        return &gpio_cpu_driver;
    }
    else {
        assert(port.dev);
        assert(port.dev->driver);
        return port.dev->driver;
    }
}
#endif

static inline void gpio_set(gpio_t gpio)
{
#ifdef MODULE_GPIO_EXP
    const gpio_driver_t *driver = gpio_driver_get(gpio.port);
    driver->set(gpio.port, (1 << gpio.pin));
#else
    gpio_cpu_set(gpio.port, (1 << gpio.pin));
#endif
}

This means that if GPIO expanders are used (module gpio_exp is enabled), the low-level API functions are always called indirectly, also for the MCU. A slight improvement in performance with slightly higher ROM requirements could be achieved if the test for the MCU port were performed separately in each function, see the approach below. In this case the RAM for the MCU driver interfacegpio_cpu_driver could also be saved. But, in the default case where GPIO expanders are not used, this doesn't matter. Therefore, in my opinion the best compromise between performance, RAM/ROM requirements and code readability is the approach above.

#if MODULE_GPIO_EXP
static inline const gpio_driver_t *gpio_driver_get(gpio_port_t port)
{
    assert(port.dev);
    assert(port.dev->driver);
    return port.dev->driver;
}
#endif

static inline void gpio_set(gpio_t gpio)
{
#ifdef MODULE_GPIO_EXP
    if (!GPIO_CPU_PORT_IS(gpio.port)) {
        const gpio_driver_t *driver = gpio_driver_get(gpio.port);
        driver->set(gpio.port, (1 << gpio.pin));
        return;
    }
#endif
    gpio_cpu_set(gpio.port, (1 << gpio.pin));
}

Benchmarks

Performance test using tests/periph_gpio and command bench 0 4:

STM32	set	clear	toggle	read	write
Master	0.166	0.187	0.416	0.187	0.208
w/o GPIO expanders (default)	0.145	0.156	0.177	0.156	0.145
with GPIO expanders (`gpio_exp` used)	0.489	0.500	0.520	0.614	0.489
ATmega 2560	set	clear	toggle	read	write
Master	4.940	5.315	10.505	5.315	5.377
w/o GPIO expanders (default)	1.876	1.938	2.188	1.750	1.876
with GPIO expanders (`gpio_exp` used)	4.565	4.627	4.877	5.440	4.565

ROM/RAM requirements for tests/pkg_semtech-loramac:

STM32	ROM	RAM	Remark
Master	49524	7936
w/o GPIO expanders (default)	49996	7968	additional RAM of 32 bytes is required, because of a complete copy of `sx127x_params` in the device variable (8 pins a 4 byte for `gpio_port_t`)
with GPIO expanders (`gpio_exp` used)	50300	7968
ATmega 2560	ROM	RAM	Remark
Master	57476	7410
w/o GPIO expanders (default)	57754	7437	additional RAM of 32 bytes is required: 16 bytes for static const pin definitions in the Harvard architecture (8 pins a 2 byte for `gpio_port_t`) 16 bytes because of a complete copy of `sx127x_params` in the device variable (8 pins a 2 byte for `gpio_port_t`)
with GPIO expanders (`gpio_exp` used)	58208	7455	additional RAM of 18 bytes is required for the MCU driver interface `gpio_cpu_driver` (9 function references a 2 bytes)

ROM/RAM requirements for tests/driver_sdcard_spi:

STM32	ROM	RAM	Remark
Master	21648	5036
w/o GPIO expanders (default)	22396	5056	additional RAM of 32 bytes is required, because of a complete copy of `sx127x_params` in the device variable (5 pins a 4 byte for `gpio_port_t`)
with GPIO expanders (`gpio_exp` used)	22436	5056
ATmega 2560	ROM	RAM	Remark
Master	23218	6080
w/o GPIO expanders (default)	23462	6104	additional RAM of 24 bytes is required: 10 bytes for static const pin definitions in the Harvard architecture (5 pins a 2 byte for `gpio_port_t`) 10 bytes because of a complete copy of `sx127x_params` in the device variable (5 pins a 2 byte for `gpio_port_t`)
with GPIO expanders (`gpio_exp` used)	23592	6116	additional RAM of 12 bytes is required for the MCU driver interface `gpio_cpu_driver` (6 function references a 2 bytes)

The additional RAM requirements expose a problem of many, if not most, of our driver implementations. They keep their own copy of the parameters in the device variable instead of simply pointing to the static constant parameters in ROM. I'm not sure if this is always the best compromise between RAM/ROM size and performance.

Testing procedure

Compilation should succeed. Tests should still work.

The new GPIO API cannot be tested standalone. The test application tests/periph_gpio_exp, which implements virtual GPIO expanders and demonstrates how to use singleport and multiport GPIO expanders, requires an MCU implementation. Such an implementation is available for ATmega in PR #14607 and for STM32 in PR #14607. Use these PRs for testing.

Issues/PRs references

Replaces PR #12877.
PR #12612 and #13925 will affect this PR and all follow-ups.

kaspar030 · 2020-07-24T12:20:15Z

Could you also provide numbers when compiled with LTO?

gschorcht · 2020-07-24T16:29:52Z

Could you also provide numbers when compiled with LTO?

Sure, I will provide them later today. I have them already for STM32, but for ATmega, collect2 of RIOT toolchain crashes.

Should I include the number in the table above or should I use a separate table?

gschorcht · 2020-09-03T07:47:06Z

I had to rebase this PR as well as PRs #14610 (STM32) and PR #14607 (ATmega) after PR #14902 was merged. Now they are all ready for a review.

maribu · 2020-11-11T16:25:49Z

I really would like to see this finally getting upstream. Maybe it is easiest to split this effort into some smaller PRs to get this less scary. How about this:

Addition of port access for peripheral GPIO ports, ideally as a static inline header only implementation for single CPU cycle access even without LTO=1
- As this would be independent of the existing periph_gpio API, this could be added one platform at a time
- This API will remain useful for high throughput bit-banging, even after a common high-level GPIO API for both external and peripheral GPIOs finally is upstream
- And having a super fast low level alternative can be a good excuse for trading in some speed for convenience in the high level GPIO API
Once all platforms have the "periph_gpio_ng", we could replace periph_gpio with a common implementation on top of periph_gpio_ng
A port access API to peripheral GPIO extender
- This could be done in parallel and fully independent of 1. / 2.
- However, the function signatures should be the same, as ultimately we want a common high level GPIO API
Once parts 1. - 3. are upstream: A common high level GPIO API
- This should be easier now, as periph_gpio is now a common platform independent implementation based on periph_gpio_ng. So only one place to modify.

Does this sound reasonable? I could help with reviewing and/or coding parts of this.

gschorcht · 2020-11-12T17:50:28Z

I really would like to see this finally getting upstream.

Me too 😉 Unfortunatly, I'm completly busy due to the second Corona online semester. That's why I didn't contribute anything to RIOT in the last months. All my contributions are currently on hold. This will probably remain so for the next two months. But I'm willing to resume my work after that.

Maybe the PR seems to be too fat, but it isn't. The core of this PR are just a small number of files.

Only some small changes in drivers/include/periph/gpio.h.
Definition of the the new GPIO API in drivers/include/periph/gpio_exp.h
CPU driver handling in drivers/periph_common/gpio.c
Test application in tests/periph_gpio_exp.

That's it. All other changes in the source code are just small modifications to make the code compilable with the new gpio_t type.

The new GPIO API is completely transparent for existing applications and the existing high-level API with one exception, comparision functions have to be used instead of value comparisons due to the structured gpio_t.

Addition of port access for peripheral GPIO ports, ideally as a static inline header only implementation for single CPU cycle access even without LTO=1

As this would be independent of the existing periph_gpio API, this could be added one platform at a time

As already mentioned, the current approach doesn't require that all platforms implement the new GPIO API at the same time. It can be done platform by platform. A platform that supports the new GPIO API just provides the feature periph_gpio_exp. If a platform doesn't support the new GPIO API, it just uses the old one.

This API will remain useful for high throughput bit-banging, even after a common high-level GPIO API for both external and peripheral GPIOs finally is upstream

And having a super fast low level alternative can be a good excuse for trading in some speed for convenience in the high level GPIO API

Once all platforms have the "periph_gpio_ng", we could replace periph_gpio with a common implementation on top of periph_gpio_ng

A port access API to peripheral GPIO extender

This could be done in parallel and fully independent of 1. / 2.

However, the function signatures should be the same, as ultimately we want a common high level GPIO API

Once parts 1. - 3. are upstream: A common high level GPIO API

This should be easier now, as periph_gpio is now a common platform independent implementation based on periph_gpio_ng. So only one place to modify.

Does this sound reasonable? I could help with reviewing and/or coding parts of this.

The critical point seems to be the new structured GPIO type which consists of a port of type gpio_port_t and pin of type gpio_pin_t. gpio_port_t is also required by the low-level API and I'm not yet sure whether it is possible to define it without the definition of the stuctured gpio_t of the high-level API.

If I understand your approach correctly, you would split drivers/include/periph/gpio_exp.h into two parts, the definition of the low-level API and the definition of the high level API that would be introduced later. Right?

stale · 2021-06-02T16:36:52Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

chrysn · 2021-08-24T12:25:09Z

AIU there's the whole gpio_abc PR tree depending on this (#12015 for ground work up to #12020 that adds WS281x support for more platforms).

@maribu, can you you process Gunar's last comments?

All involved, other than questions of whether to split this into one or two components, is there any reason this should generally not be done? (I don't think so, just trying to paint the roadmap here).

maribu · 2021-08-25T07:43:27Z

I currently need a ws281x driver for the nRF52 family and reconsidered the approach. Basically, a fast (1-2 CPU cycles) GPIO API that doesn't rely on LTO and parameters being correctly identified as compile time constants would allow ignoring the overhead of the GPIO API for the bit-banging timing. Adding a separate API for precise busy-waiting for short durations (maybe using the SysTick timer on Cortex-M rather than counting CPU cycles to also work for Cortex M7 and not having to experimentally the number of CPU cycles per loop iteration) would do the trick.

I'm currently working on a new low-level peripheral GPIO API that is addressing this and a couple of other limitations of the current GPIO API, which is loosely based on this PR. It will however not take GPIO extenders into account. The current pin-oriented API can easily be implemented on top of that at one place - and likely most users will stick with the current API. It should be way easier to hook GPIO extenders into the pin-oriented GPIO API once there is only a single, hardware independent implementation of it.

The low-level port-oriented GPIO API that I'm working on is IMO not the right place for hooking in GPIO extenders, as this would affect timing. If we want to do precise bit-banging as required for speaking to the ws281x, we are limited to using the peripheral GPIOs and depend on reliable timing of GPIO accesses. Use-cases not depending on this will likely prefer the pin-based API anyway and will work just fine for external GPIOs.

chrysn · 2021-08-25T07:50:57Z

ws281x driver for the nRF52 family

Just as a data point, that was relatively easy to achieve also based on #12015 which is still based on the current GPIO API.

maribu · 2021-08-25T08:23:47Z

ws281x driver for the nRF52 family

Just as a data point, that was relatively easy to achieve also based on #12015 which is still based on the current GPIO API.

Well, GPIO ABC is a new API just for bit-banging that is only needed because the current API is too slow. And we still are not able to bit-bang parallel protocols with it, as this would inherently need a port based API. (And also: The current approach of counting CPU cycles in GPIO ABC relies on the same code taking the same amount of CPU cycles. But with the dynamic branch predictor the delay loop on STM32F7 takes easier 1 CPU cycles for the two instructions (using the dual issue feature), or something in the order of 10 CPU cycles if the pipeline stalls. So we need to come up with something else for at least some boards anyway.)

Also the GPIO initialization API IMO sucks, as we cannot expose features commonly present on modern MCUs easily. E.g. if an MCU as strong and weak internal pull resistors and can not only pull and down, but also keep the input at bus level (pull to whatever the input was last, so that it needs to be externally driven to the opposite level) we would get a lot of gpio_mode_t constants. Most MCUs also can set the driving strength, enable or disable Schmitt triggers, optimize inputs for high or low data rate, etc. I have also some use cases where I would like to initialize a GPIO with an initial value (e.g. to directly go from high impedance to output at level high, rather than shortly pushing an externally pulled up signal low during initialization).

So there are actually plenty of reasons I dislike the current API. The ws281x driver is just an excuse to finally scratch that itch.

stale · 2022-03-02T09:18:30Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

gschorcht requested review from aabadie, basilfx, fjmolinas, haukepetersen, jia200x, leandrolanzieri, MichelRottleuthner, miri64, MrKevinWeiss and smlng as code owners July 24, 2020 11:39

gschorcht requested review from maribu, benpicco and kaspar030 July 24, 2020 11:39

gschorcht force-pushed the periph/gpio_exp_v5b/api branch from d36d760 to daf5f11 Compare July 24, 2020 11:57

gschorcht mentioned this pull request Jul 24, 2020

[POC/RFC/WIP] periph/gpio: Proposal for the change of the GPIO API #12877

Closed

This was referenced Jul 24, 2020

cpu/atmega: MCU implementation of the new GPIO API #14607

Open

cpu/stm32: MCU implementation of the new GPIO API #14610

Open

gschorcht added State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet and removed State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet labels Jul 24, 2020

gschorcht mentioned this pull request Jul 25, 2020

ports/riot: change for structured GPIO type kaspar030/micropython#3

Open

gschorcht force-pushed the periph/gpio_exp_v5b/api branch from 1d35ec4 to 70db5f7 Compare July 26, 2020 15:03

gschorcht added 3 commits September 3, 2020 08:55

fixup! drivers/periph: new expandable GPIO API

e03a4c2

pkg: required changes for new GPIO API

51d7587

fixup! drivers/periph: new expandable GPIO API

9024fba

gschorcht force-pushed the periph/gpio_exp_v5b/api branch from 9b5b336 to 9024fba Compare September 3, 2020 07:28

gschorcht added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed State: waiting for other PR State: The PR requires another PR to be merged first labels Sep 3, 2020

gschorcht added 2 commits September 3, 2020 09:50

fixup! drivers/periph: new expandable GPIO API

08a7a34

drivers: use inline functions for GPIO comparisons

ad7c4d1

maribu mentioned this pull request Feb 5, 2021

Introduce ATxmega CPU and Boards #15758

Merged

gschorcht mentioned this pull request Feb 27, 2021

drivers: add MCP23x17 I/O expander support #10688

Merged

chrysn mentioned this pull request Mar 21, 2021

drivers/periph: Added gpio_abc (GPIO Advanced Bitbanging Capabilities) #12015

Closed

jeandudey mentioned this pull request Mar 22, 2021

drivers/soft_spi_as_periph_spi: new module #16196

Open

1 task

stale bot added the State: stale State: The issue / PR has no activity for >185 days label Jun 2, 2021

gschorcht removed the State: stale State: The issue / PR has no activity for >185 days label Jun 2, 2021

MrKevinWeiss added this to the Release 2021.07 milestone Jun 21, 2021

MrKevinWeiss removed this from the Release 2021.07 milestone Jul 15, 2021

maribu mentioned this pull request Aug 31, 2021

periph/gpio_ll: New Peripheral GPIO API #16787

Merged

stale bot added the State: stale State: The issue / PR has no activity for >185 days label Mar 2, 2022

chrysn mentioned this pull request Mar 2, 2022

drivers/ws281x: Add GPIO ABC backend #12020

Closed

gschorcht added State: don't stale State: Tell state-bot to ignore this issue and removed State: stale State: The issue / PR has no activity for >185 days labels Mar 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

periph/gpio: new expandable GPIO API #14602

periph/gpio: new expandable GPIO API #14602

Uh oh!

gschorcht commented Jul 24, 2020 •

edited

Loading

Uh oh!

kaspar030 commented Jul 24, 2020

Uh oh!

gschorcht commented Jul 24, 2020

Uh oh!

gschorcht commented Sep 3, 2020 •

edited

Loading

Uh oh!

maribu commented Nov 11, 2020

Uh oh!

gschorcht commented Nov 12, 2020

Uh oh!

stale bot commented Jun 2, 2021

Uh oh!

chrysn commented Aug 24, 2021

Uh oh!

maribu commented Aug 25, 2021

Uh oh!

chrysn commented Aug 25, 2021

Uh oh!

maribu commented Aug 25, 2021 •

edited

Loading

Uh oh!

stale bot commented Mar 2, 2022

Uh oh!

Uh oh!

periph/gpio: new expandable GPIO API #14602

Are you sure you want to change the base?

periph/gpio: new expandable GPIO API #14602

Uh oh!

Conversation

gschorcht commented Jul 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution description

Implementation Details

GPIO pins

GPIO ports

MCU Low-level API Implementation

How the new GPIO API works

Benchmarks

Testing procedure

Issues/PRs references

Uh oh!

kaspar030 commented Jul 24, 2020

Uh oh!

gschorcht commented Jul 24, 2020

Uh oh!

gschorcht commented Sep 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maribu commented Nov 11, 2020

Uh oh!

gschorcht commented Nov 12, 2020

Uh oh!

stale bot commented Jun 2, 2021

Uh oh!

chrysn commented Aug 24, 2021

Uh oh!

maribu commented Aug 25, 2021

Uh oh!

chrysn commented Aug 25, 2021

Uh oh!

maribu commented Aug 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stale bot commented Mar 2, 2022

Uh oh!

Uh oh!

gschorcht commented Jul 24, 2020 •

edited

Loading

gschorcht commented Sep 3, 2020 •

edited

Loading

maribu commented Aug 25, 2021 •

edited

Loading