docs: Create a Hardware Integration index page (#2634)

Co-authored-by: Nicolas Munnich <[email protected]>

5 files changed, 164 insertions, 82 deletions

diff --git a/docs/docs/development/hardware-integration/boards-shields-keymaps.md b/docs/docs/development/hardware-integration/boards-shields-keymaps.md
deleted file mode 100644
index c6e141b9aa..0000000000
--- a/docs/docs/development/hardware-integration/boards-shields-keymaps.md
+++ /dev/null
@@ -1,52 +0,0 @@
----
-title: Boards, Shields, and Keymaps
----
-
-## Architecture Overview
-
-The foundational elements needed to get a specific keyboard working with ZMK can be broken down into:
-
-- A [KSCAN driver](https://docs.zephyrproject.org/3.5.0/reference/peripherals/kscan.html), which uses `compatible="zmk,kscan-gpio-matrix"` for GPIO matrix based keyboards, or uses `compatible="zmk,kscan-gpio-direct"` for small direct wires.
-- An optional matrix transform, which defines how the KSCAN row/column events are translated into logical "key positions". This is required for non-rectangular keyboards/matrices, where the key positions don't naturally follow the row/columns from the GPIO matrix.
-- A keymap, which binds each key position to a behavior, e.g. key press, mod-tap, momentary layer, in a set of layers.
-
-These three core architectural elements are defined per-keyboard, and _where_ they are defined depends on the specifics of how that
-keyboard works. For an overview on the general concepts of boards and shields, please see the [FAQs on boards and shields](../../faq.md#why-boards-and-shields-why-not-just-keyboard).
-
-## Self-Contained Keyboard
-
-For a self-contained keyboard that includes the microprocessor, all of the above architecture components are included in the Zephyr _board_ definition. You can see an example for the [Planck V6](https://github.com/zmkfirmware/zmk/tree/main/app/boards/arm/planck) board directory.
-
-With this type of keyboard, the full ZMK definition for the keyboard exists
-in the `app/boards/${arch}/${board_name}` directory, e.g. `app/boards/arm/planck/`. In that directory, you'll have the following:
-
-- A `Kconfig.board` file that defines the toplevel Kconfig item for the board, including which SoC Kconfig setting it depends on.
-- A `Kconfig.defconfig` file that sets some initial defaults when building this keyboard. This usually includes:
-  - Setting `ZMK_KEYBOARD_NAME` to a value, for the product name to be used for USB/BLE info.
-  - Setting `ZMK_USB` and/or `ZMK_BLE` for the default values for which HID transport(s) to enable by default
-- A `${board_name}_defconfig` file that forces specific Kconfig settings that are specific to this hardware configuration. Mostly this is SoC settings around the specific hardware configuration.
-- `${board_name}.dts` which contains all the devicetree definitions, including:
-  - An `#include` line that pulls in the specific microprocessor that is used, e.g. `#include <st/f3/stm32f303Xc.dtsi>`.
-  - A [chosen](https://docs.zephyrproject.org/3.5.0/build/dts/intro-syntax-structure.html#aliases-and-chosen-nodes) node named `zmk,kscan` which references the configured KSCAN driver (usually a GPIO matrix)
-  - (Optional) A [chosen](https://docs.zephyrproject.org/3.5.0/build/dts/intro-syntax-structure.html#aliases-and-chosen-nodes) node named `zmk,matrix-transform` that defines the mapping from KSCAN row/column values to the logical key position for the keyboard.
-- A `board.cmake` file with CMake directives for how to flash to the device.
-- A `${board_name}.keymap` file that includes the default keymap for that keyboard. Users will be able to override this keymap in their user configs.
-
-## Pro Micro Compatible Keyboard
-
-![Labelled Pro Micro pins](../../assets/interconnects/pro_micro/pinout.png)
-
-For keyboards that require a (usually Pro Micro compatible) add-on board to operate, the ZMK integration pieces are places
-in the _shield_ definition for that keyboard, allowing users to
-swap in different Pro Micro compatible boards (e.g. Proton-C, or nice!nano) and build a firmware the matches their actual
-combination of physical components.
-
-With this type of keyboard, the partial definition for the keyboard exists
-in the `app/boards/shields/${board_name}` directory, e.g. `app/boards/shields/clueboard_california/`. In that directory, you'll have the following:
-
-- A `Kconfig.shield` that defines the toplevel Kconfig value for the shield, which uses a supplied utility to function to default the value based on the shield list, e.g. `def_bool $(shields_list_contains,clueboard_california)`.
-- A `Kconfig.defconfig` file to set default values for things like `ZMK_KEYBOARD_NAME`
-- A `${shield_name}.overlay` file, which is a devicetree overlay file, that includes:
-  - A [chosen](https://docs.zephyrproject.org/3.5.0/build/dts/intro-syntax-structure.html#aliases-and-chosen-nodes) node named `zmk,kscan` which references the configured KSCAN driver (usually a GPIO matrix). For these keyboards, to be compatible with any Pro Micro compatible boards, the KSCAN configuration should reference the [nexus node](https://docs.zephyrproject.org/3.5.0/hardware/porting/shields.html#gpio-nexus-nodes) that ZMK has standardized on. In particular, the `&pro_micro` aliases can be used to reference the standard digital pins of a Pro Micro for shields.
-  - (Optional) A [chosen](https://docs.zephyrproject.org/3.5.0/build/dts/intro-syntax-structure.html#aliases-and-chosen-nodes) node named `zmk,matrix-transform` that defines the mapping from KSCAN row/column values to the logical key position for the keyboard.
-- A `keymap/keymap.overlay` file that includes the default keymap for that keyboard. Users will be able to override this keymap in their user configs.
diff --git a/docs/docs/development/hardware-integration/index.mdx b/docs/docs/development/hardware-integration/index.mdx
new file mode 100644
index 0000000000..efde1081d1
--- /dev/null
+++ b/docs/docs/development/hardware-integration/index.mdx
@@ -0,0 +1,155 @@
+---
+title: Hardware Integration
+sidebar_label: Overview
+---
+
+import Tabs from "@theme/Tabs";
+import TabItem from "@theme/TabItem";
+
+This section of the documentation describes steps necessary to get ZMK running on a keyboard, including basic keyboard functionality as well as additional features such as encoders.
+Please see pages in the sidebar for guides and reference that describe different aspects of this integration.
+
+The foundational elements needed to get a specific keyboard working with ZMK can be broken down into:
+
+- A [physical layout](physical-layouts.md) that describes the electrical and physical structure of the keyboard, referring to:
+  - A [kscan driver](../../config/kscan.md), which most frequently uses `compatible = "zmk,kscan-gpio-matrix"` for GPIO matrix based keyboards, or `compatible = "zmk,kscan-gpio-direct"` for direct wires.
+  - A [matrix transform](../../config/layout.md), which defines how the kscan row/column events are translated into logical "key positions".
+  - An [optional description](physical-layouts.md#optional-keys-property) of physical key positions and sizes, in order to visualize the keyboard accurately in [ZMK Studio](../../features/studio.md).
+- A [keymap](../../keymaps/index.mdx), which binds each key position to a behavior, e.g. key press, mod-tap, momentary layer, in a set of layers.
+- Other, optional configuration items to support features such as encoders or lighting systems.
+
+These core architectural elements are defined per-keyboard, and _where_ they are defined depends on the specifics of how that keyboard is structured.
+
+## Boards & Shields
+
+ZMK uses the Zephyr concepts of "boards" and "shields" to refer to different parts of a keyboard build, that in turn get combined during a firmware build.
+Also see the Zephyr documentation on [boards](https://docs.zephyrproject.org/3.5.0/glossary.html#term-board) and [shields](https://docs.zephyrproject.org/3.5.0/hardware/porting/shields.html).
+
+### What is a "board"?
+
+In ZMK, a _board_ defines the _PCB_ that _includes the microcontroller unit (MCU)_.
+For keyboards, this is one of two options:
+
+- Complete keyboard PCBs that include the MCU (e.g. the Planck or Preonic).
+- Small MCU boards (e.g. the nice!nano or Seeed Studio Xiao RP2040) that expose pins and are designed to be combined with larger keyboard PCBs, or hand-wired to switches to create the final keyboard.
+
+### What is a "shield"?
+
+In ZMK, a _shield_ is a _PCB_ or _hardwired set of components_ that when combined with an MCU-only board, like the SparkFun Pro Micro RP2040 or nice!nano, results in a complete usable keyboard.
+Examples would be keyboard PCBs like the Kyria or Lily58.
+The shield is usually the big PCB containing all the keys.
+
+### Why not just "keyboard"?
+
+["Composite" keyboards](../../hardware.mdx#composite) are keyboards which use a separate small PCB MCU module for its "brains".
+In such keyboards, the [shield](#what-is-a-shield) is a brainless shell containing all the keys, RGB LEDs, encoders etc.
+It typically maps all of these features to a standard pin footprint, such as the Pro Micro pinout.
+
+To bring this brainless shield to life, you attach any MCU [board](#what-is-a-board) matching the footprint.
+For instance, the _nice!nano_ is _pin-compatible_ with the _SparkFun Pro Micro RP2040_, so you can substitute either board onto the shield.
+But each board comes with its own features (MCU, flash, BLE, etc.) which must also be handled.
+
+Therefore in ZMK, board and shield are considered two different (but related) entities so that it is easier to mix and match them, and they are combined during a ZMK build.
+This provides the modularity to be able to use composite keyboards with different compatible controllers.
+
+Note that ["self-contained" keyboards](../../hardware.mdx#onboard) only have a single PCB which includes the "brains" (MCU) onboard.
+In ZMK, these have no shield, only a board.
+
+## Organization Overview
+
+<Tabs
+defaultValue="self-contained"
+values={[
+{label: 'Self-contained keyboards', value: 'self-contained'},
+{label: 'Composite keyboards', value: 'composite'},
+]}>
+
+<TabItem value="self-contained">
+
+For a [self-contained keyboard](../../hardware.mdx#onboard) that includes the microprocessor, all of the above architecture components are included in the Zephyr _board_ definition and no shield is defined.
+You can see an example for the [Planck V6](https://github.com/zmkfirmware/zmk/tree/main/app/boards/arm/planck) board directory.
+
+With this type of keyboard, the full ZMK definition for the keyboard exists in the `<board_root>/boards/<arch>/<keyboard_name>` directory where `<board_root>` is `zmk/app` or a [module](../../features/modules.mdx) root, e.g. `zmk/app/boards/arm/planck/`.
+In that directory you'll have the following files, where there can be multiples of files with `<board_name>`s, corresponding to each keyboard part for [split keyboards](../../features/split-keyboards.md):
+
+```
+<keyboard_name>
+├── Kconfig.board
+├── Kconfig.defconfig
+├── <board_name>_defconfig
+├── <board_name>.dts
+├── <keyboard_name>.keymap
+├── board.cmake
+└── <keyboard_name>.zmk.yml
+```
+
+These files include [base Kconfig files](https://docs.zephyrproject.org/3.5.0/build/kconfig/index.html):
+
+- A `Kconfig.board` file that defines the toplevel [Kconfig](https://docs.zephyrproject.org/3.5.0/build/kconfig/index.html) items for the board, including which SoC Kconfig setting it depends on.
+- A `Kconfig.defconfig` file that sets some initial defaults when building this keyboard. This usually includes:
+  - Setting [`ZMK_KEYBOARD_NAME`](../../config/system.md#general) to a value, for the product name to be used for USB/BLE info,
+  - Setting [`ZMK_USB`](../../config/system.md#usb) and/or [`ZMK_BLE`](../../config/system.md#bluetooth) for the default values for which HID transport(s) to enable by default
+
+[Configuration files](../../config/index.md#kconfig-files) that set the visible Kconfig symbols:
+
+- A `<board_name>_defconfig` file that forces specific Kconfig settings that are specific to this hardware configuration.
+  These are mostly SoC settings around the specific hardware configuration.
+
+[Devicetree files](../../config/index.md#devicetree-files):
+
+- `<board_name>.dts` which contains all the devicetree definitions[^1], including but not limited to:
+  - An `#include` line that pulls in the specific microprocessor that is used, e.g. `#include <st/f3/stm32f303Xc.dtsi>`,
+  - Kscan, matrix transform and physical layout devicetree nodes as described above,
+  - A [chosen](https://docs.zephyrproject.org/3.5.0/build/dts/intro-syntax-structure.html#aliases-and-chosen-nodes) node including `zmk,physical-layout` property among others, where each property references the nodes defined in the file.
+- A `<keyboard_name>.keymap` file that includes the default keymap for that keyboard. Users will be able to override this keymap in their user configs.
+
+And other miscellaneous ones:
+
+- A `board.cmake` file with CMake directives for how to flash to the device.
+- A `<keyboard_name>.zmk.yml` file containing [metadata](hardware-metadata-files.md) for the keyboard.
+
+See Zephyr's [board porting guide](https://docs.zephyrproject.org/3.5.0/hardware/porting/board_porting.html) for information on creating a new board.
+Also see the [new keyboard shield guide](new-shield.mdx#shield-overlays) for information on parts of the devicetree specifically related to ZMK.
+
+[^1]:
+    Parts of these files can live in separate `.dtsi` files (typically in the same directory) that are then `#include`d in the files, to reduce duplication or improve organization.
+    For instance, a shared `<keyboard_name>.dtsi` file is used for split keyboards that contains devicetree definitions that are shared across keyboard parts.
+
+</TabItem>
+<TabItem value="composite">
+
+Keyboards that require an add-on board to operate are [composite keyboards](../../hardware.mdx#composite), where the ZMK integration pieces are placed in the _shield_ definition for that keyboard.
+This allows users to swap in different boards that use the same interconnect (e.g. Pro Micro RP2040, or nice!nano) and build a firmware the matches their actual combination of physical components.
+
+With this type of keyboard, the partial definition for the keyboard exists in the `<board_root>/boards/shields/<keyboard_name>` directory where `<board_root>` is `zmk/app` or a [module](../../features/modules.mdx) root, e.g. `zmk/app/boards/shields/clueboard_california/`.
+In that directory, you'll have the following files, where there can be multiple `<shield_name>`s, corresponding to each keyboard part for [split keyboards](../../features/split-keyboards.md):
+
+```
+<keyboard_name>
+├── Kconfig.shield
+├── Kconfig.defconfig
+├── <shield_name>.overlay
+├── <keyboard_name>.keymap
+└── <keyboard_name>.zmk.yml
+```
+
+These files include [base Kconfig files](new-shield.mdx#base-kconfig-files):
+
+- A `Kconfig.shield` that defines the toplevel Kconfig value for the shield, which uses a supplied utility to function to default the value based on the shield list, e.g. `def_bool $(shields_list_contains,clueboard_california)`.
+- A `Kconfig.defconfig` file to set default values for settings like `ZMK_KEYBOARD_NAME`
+
+[Devicetree files](../../config/index.md#devicetree-files):
+
+- A `<shield_name>.overlay` file which is a devicetree overlay file[^1], containing definitions including but not limited to:
+  - Kscan, matrix transform and physical layout devicetree nodes as described above, where the kscan node uses the interconnect [nexus node](https://docs.zephyrproject.org/3.5.0/hardware/porting/shields.html#gpio-nexus-nodes) aliases such as `&pro_micro` for GPIO pins.
+  - A [chosen](https://docs.zephyrproject.org/3.5.0/build/dts/intro-syntax-structure.html#aliases-and-chosen-nodes) node including at least the `zmk,physical-layout` property, referring to the defined node.
+- A `<keyboard_name>.keymap` file that includes the default keymap for that keyboard. Users will be able to override this keymap in their user configs.
+
+And other miscellaneous ones:
+
+- A `<keyboard_name>.zmk.yml` file containing [metadata](hardware-metadata-files.md) for the keyboard.
+
+See the [new keyboard shield guide](new-shield.mdx) for documentation that walks you through creating these files to define a new composite keyboard.
+
+</TabItem>
+</Tabs>
diff --git a/docs/docs/faq.md b/docs/docs/faq.md
index 901e9c965f..79fc0af8c5 100644
--- a/docs/docs/faq.md
+++ b/docs/docs/faq.md
@@ -47,34 +47,11 @@ Sorry, Zephyr™ only supports 32-bit and 64-bit platforms.
 
 ZMK is still in its infancy, so there’s a learning curve involved. But if you’d like to try it out, please check out the development [documentation](/docs) and the other FAQs. Please keep in mind that the project team is still small, so our support capability is limited whilst we focus on development. But we’ll try our best! Interested developers are also very welcome to contribute!
 
-### What is a “board”?
+### What are _boards_ and _shields_? Why not just "keyboard"?
 
-In ZMK, a _board_ defines the _PCB_ that _includes the MCU_.
-For keyboards, this is one of two options:
-
-- Complete keyboard PCBs that include the MCU (e.g. the Planck or Preonic).
-- Small MCU boards (e.g. the Proton-C or nice!nano) that expose pins and are designed to be combined with larger keyboard PCBs, or hand wired to switches to create the final keyboard.
-
-### What is a “shield”?
-
-In ZMK, a _shield_ is a _PCB_ or _hardwired set of components_ that when combined with an MCU-only [board](#what-is-a-board), like the Proton-C or nice!nano, results in a complete usable keyboard. Examples would be keyboard PCBs like the Kyria or Lily58. The _shield_ is usually the big PCB containing all the keys.
-
-### Why _boards_ and _shields_? Why not just “keyboard”?
-
-If you haven't already done so, please read these FAQs first:
-
-- [What is a “board”?](#what-is-a-board)
-- [What is a "shield"?](#what-is-a-shield)
-
-When a keyboard accepts a small “PCB MCU module” (e.g. _Arduino Pro Micro_) for its “brains”, then it's important to conceptually separate the hardware into a [board](#what-is-a-board) PCB and a [shield](#what-is-a-shield) PCB.
-
-The [shield](#what-is-a-shield) is a brainless shell containing all the keys, RGB LEDs, encoders etc. It maps all of these features to a standard pin footprint, such as the Pro Micro pinout.
-
-To bring this brainless [shield](#what-is-a-shield) to life, you attach any MCU [board](#what-is-a-board) matching the footprint. For instance, the _nice!nano_ is _pin-compatible_ with the _Arduino Pro Micro_, so you can substitute either [board](#what-is-a-board) onto the [shield](#what-is-a-shield). But each [board](#what-is-a-board) comes with its own features (MCU, flash, BLE, etc.) which must also be handled.
-
-Therefore in ZMK, [board](#what-is-a-board) and [shield](#what-is-a-shield) are considered two different (but related) entities so that it’s easier to mix and match them. They are combined during a ZMK build.
-
-Please note, many keyboards only have a single PCB which includes the “brains” (MCU) onboard. In ZMK, these have no [shield](#what-is-a-shield), only a [board](#what-is-a-board).
+ZMK uses the Zephyr concepts of "boards" and "shields" to refer to different parts of a keyboard build, that in turn get combined during a firmware build.
+This provides the modularity to be able to use composite keyboards with different compatible controllers.
+Please see the [explainer on boards & shields](development/hardware-integration/index.mdx#boards--shields) for more details.
 
 ### Does ZMK support wired split?
 
diff --git a/docs/docs/hardware.mdx b/docs/docs/hardware.mdx
index 09f98c84eb..75209923fa 100644
--- a/docs/docs/hardware.mdx
+++ b/docs/docs/hardware.mdx
@@ -41,8 +41,7 @@ export const toc = [
 ];
 
 With the solid technical foundation of Zephyr™ RTOS, ZMK can support a wide diversity of hardware targets.
-That being said, there are specific [boards](faq.md#what-is-a-board) /
-[shields](faq.md#what-is-a-shield) that have been implemented and tested by the ZMK contributors, listed below.
+That being said, there are specific [boards / shields](development/hardware-integration/index.mdx#boards--shields) that have been implemented and tested by the ZMK contributors, listed below.
 
 <HardwareList items={Metadata} />
 
diff --git a/docs/sidebars.js b/docs/sidebars.js
index e7975fb2d5..c480a90721 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -124,12 +124,15 @@ module.exports = {
         {
           type: "category",
           label: "Hardware Integration",
+          link: {
+            type: "doc",
+            id: "development/hardware-integration/index",
+          },
           collapsed: true,
           items: [
             "development/hardware-integration/new-shield",
             "development/hardware-integration/physical-layouts",
             "development/hardware-integration/hardware-metadata-files",
-            "development/hardware-integration/boards-shields-keymaps",
             "development/hardware-integration/pinctrl",
             "development/hardware-integration/shift-registers",
             "development/hardware-integration/encoders",