Each custom register handler has its own almost similar code body.
Can I simplify this into one function for writing, one for reading registers?

Avoids duplicate logic (and duplicate code lines, although that's not the driver)

adc with ADS1115:

for RAW adc return:
{.pattern = "ANAlog:16:INPut#:RAW?", .callback = SCPI_Analog_ADS1115_InputQ,},

optional, not needed but can be considered: Volt return:
{.pattern = "ANAlog:16:INPut#[:VOLt]?", .callback = SCPI_Analog_ADS1115_VoltInputQ,},

init functionality to be added to initInstrument() in scpi-def.c

A new system temperature module to support the measurement of either ambient temperature of the system, or board temperature of individual cards.

usb init does not work in SDK 1.5 version of tinyusb

  // init device stack on configured roothub port
  tud_init(BOARD_TUD_RHPORT);

  while (1)

ANALOG:OUTP0:RAW to set the raw duty cycle value of the first PWM pin in array of pwm pins

API to reside in src/pwm
look for placeholders and TODO in scpi-def.c and pwm_utils.c
attention: template code has uint32_t for duty cycle, let's use a uint16_t, in line with pico pwm api

pwm_set_gpio_level(gpio, x)

Merge branch feature-set-2 to main when feature set 2 is complete
scope:
#20
#6

pair with a labview driver reelase

Introduction

PST would benefit from the ability for software features to be able to store and retrieve bits of information. That information could likely be calibration data for on-board or attached hardware, but could conceivably be configuration information, passwords, and so on, for future features. The Data Storage (DS) capability for PST will provide support for software developers who wish to use it for their features.

Vocabulary

For the purposes of following this document, the following definitions/concepts will be used:

Block: Flash storage is in chunks we will call Blocks (e.g. a 4 kbyte Block of Flash)

DS: The overall capability described in this document will be called DS (Data Storage)

Item: An Item is a single value or variable that needs DS capability. For instance, an offset integer used by an ADC feature would be an Item.

Group: A Group will be expected to be smaller than a Block; it will always be of a size in multiples of 16. For the MVP, there will be a single Group per feature that needs DS capability. As an example, if there is a feature called HIRES that needs 15 bytes of DS capability, then the HIRES DS Group will be of a size of (perhaps) 32 bytes (larger than 16, so that there is room for the HIRES feature to comfortably expand its DS needs in future), and that Group will contain a C Structure (described next).

Group data Structure: The Group data structure will be accessible in RAM, and will (when things are consistent) be a copy of the
Group that is within the Block in Flash. The structure will contain things such as a version number and the actual data content (in types such as integer, float and C string), which are the Items. The C structure will always be of a smaller size than the Group size; the C structure is effectively the data-aware interpretation of the group.

High-Level Requirements

As a software developer, access to an API for storing and retrieving data should be available, so that it is possible to maintain persistent, non-volatile information in an easy-to-use way.

A software developer would like to be able to, at a minimum, store and retrieve types of data such as calibration integers, or floating-point values, or strings, so that the end feature can easily incorporate the data items.

A software developer must be able to develop features that can call an API to determine if data is valid or not, so that unexpected values are not used by their feature.

A software developer must be able to store default data if the current stored data is invalid, so that a sane configuration can be made available for new device or new feature and for corrupted memory circumstances.

High-Level Design

Ability for the developer to allocate storage for a feature during compile-time

There should be room available for features to grow in future releases, without trampling on storage allocated for other features.
One approach could be to statically define indexes for the storage, in an array, with one entry per feature. For instance, if two features called HIRES and PWM required storage, then two DS groups would be defined:

#define DS_GROUP_HIRES 0
#define DS_GROUP_PWM 1
#define DS_GROUP_UNALLOCATED 2

// allocate a minimum of 32 bytes, and ensure when adding a new entry, that at least 16 bytes are spare for subsequent enhancements for the feature! And all index values must be on 16 byte boundaries. Example: A value of 18 is _not_valid_! 

uint16_t ds_mem_index[] = {0 /* HIRES */, 
          32 /* PWM */,
         64 /* UNALLOCATED */};

Anyone creating a new feature would move the unallocated value.

Note: The linker file may need to be adapted, to avoid that the area is ever overwritten by growing firmware size. GCC allows for it. Optional: it also allows to use the address defined in the linker file in C code._Check the Pico SDK to see what is suggested. Today the Pico SDK examples deliberately choose a block that is right at the end of Flash, so that the code would need to fill all remainder Flash space before that block could be accidentally overwritten.
A separate user story is created for it.

There would be a base address for internal flash, for instance:

#define DS_MEM_FLASH_START 0xffff1000

(edit jc: @shabaz123, I added this area))
issue 34 created a symbol for this area in the linker / loader script. In C code, we can get at the start address via:

in our DS api header, if we write this code (takes no memory space or clock ticks. It is as efficient as using a define)

// only if no pico headers used - normally not required because we'll include the flash headers #include <stdint.h>

inline uint32_t *ds_get_address_persistent() {
    extern uint32_t ADDR_PERSISTENT[];
    return ADDR_PERSISTENT;
}

When the base address is needed, we can use this (example code)
sprintf(addr, "address = %x", ds_get_address_persistent());

Ability for the developer to define the data

This could be in the form of a C structure, for instance:

typedef struct ds_hires_group_s {
  int version;  
  int16_t offset;
  float gradient;
} ds_hires_group_t;

A version value should be mandatory, perhaps integer type.

For a MVP, all persistent data could be copied in RAM, i.e. global variables:

ds_hires_group_t ds_hires_data;

To make it easier to access the data, definitions could be written, for instance:

#define DS_ITEM_HIRES_OFFSET 0
#define DS_ITEM_HIRES_GRADIENT 1
#define DS_ITEM_PWM_OFFSET 0
#define DS_ITEM_PWM_FREQ 1

Ability for the developer to get and store data

Access to the data should ideally be through function calls, instead of directly to the RAM copy. If direct access to the RAM copy was allowed, then in the future it would be hard to make improvements to the mechanism.

Example function calls could be:

void ds_store_item(int itemnum, void* valptr);
void* ds_get_item(int itemnum);

Storing an item:

ds_store_item(DS_ITEM_HIRES_OFFSET, (void*)&offset);

Retrieving an item:

offset = *((int*)ds_get_item(DS_ITEM_HIRES_OFFSET));

These functions would manipulate the RAM copy (i.e. it wouldn’t immediately write to Flash). The ds_store_item function would need to set a flag to indicate that the RAM copy is different to Flash, so that persistent storage is only written to when a change occurs.

The flag could be called (say) ds_nv_write_pending.

The ds_store_item function should check if the new value is the same as existing data, to prevent unnecessary flash writes to a location. The number of writes is limited. The ds_nv_write_pending flag should only be set if any data is different to what is in Flash.

Ability for the RAM copy to be initialized from persistent storage

The initXYZUtils() function for features that use the DS capability should be responsible for calling a function, for instance as follows:

ds_mem_init(DS_GROUP_HIRES);

That function would use (in the MVP) perhaps memcpy, to copy the chunk of Flash into the RAM copy as a block, so that then the ds_get_item function can be used at any time. In future, an enhancement to the ds_mem_init function could be to read from I2C memory if it is present.

The ds_mem_init function should also be responsible for doing a sanity check (at a minimum ensuring a magic number (which can be global, not feature-specific) is present and the version number is correct), and if not, then default values should be written to RAM, and the ds_nv_write_pending flag should be set.

Ability for the RAM copy of all feature storage to be written to persistent storage

Even though each feature may only use (say) 32 or 48 or 64 bytes and so on, of storage, it is not possible to write such a small group of data to Flash; there will be a Flash block size, for instance 4 kbytes. So, all the features data storage (groups) can be written in one go.

At the end of the initInstrument() function (in scpi-def.c), it should be possible to have a function call, such as ds_write_persistent_all(), which will write all the groups into Flash. The function will only do this if the ds_nv_write_pending flag is set, and then the flag would be cleared.

Ability for the developer to write to feature storage at any time

Whenever a feature requires data to become persistent, a function call should be executed, called (say):

ds_write_persistent_group(DS_GROUP_HIRES);

That function could, for the MVP, simply call the ds_write_persistent_all() function.

(jc edit)

possible SCPI commands (suggestion)

//  infra
    {.pattern = "CALibration:STArt", .callback = calStart,},
    {.pattern = "CALibration:END", .callback = calEnd,},
    {.pattern = "CALibration:ERAse", .callback = calErase,},

// funtional (examples to show possible signature)
    {.pattern = "CALibration:ADC#:VOLTage?", .callback = calAdcVoltQ,},
    {.pattern = "CALibration:TEMPERATUREMAXResistance", .callback = calTemperatureMaxResistance,},

at this moment, the function sets the service request flag.
It makes more sense to let it do the same as *TRG

all functionality that can be shared in lablib
all instrument code in labtool

makes it easier to write multiple small instruments

allow users to register an event on gpio in
set register when event occured

NI doco:
Using Instrument Status Registers and Service Requests in LabVIEW
Gough blog:
Tutorial: Understanding the SCPI Status Model with pyvisa Examples
omicron doco:
Event Based Status Register Groups
Jan Breuer code section for custom registers:
github link
MCS
MCS analyser SCPI manual p. 256
EEZ
programmable PSU manual
copper mountain tech
good register use drawing

i2c init is now in the ads1115 code.
move it outside, and let the core instrument initialise it.
this will help in case an other i2c IC is used, or if 2 i2c ICs are attached

link to code

find way to reserve block(s) of flash space, so that it can't be overwritten by new, larger, firmware

• - look for existing projects with custom linker / loader script
• - learn
• - make proof of concept
• - adapt to code base

sdk doc with linker info.

pico_standard_link encapsulates the standard linker setup needed to configure the type of application binary layout in
memory, and link to any additional C and/or C++ runtime libraries. It also includes the default crt0, which provides the
initial entry point from the flash second stage bootloader, contains the initial vector table (later relocated to RAM), and
initialises static data and RAM-resident code if the application is running from flash.

custom linker file:
pico_set_linker_script(my_target ${CMAKE_CURRENT_LIST_DIR}/custom.ld)

default loader source.

sdk linker settings

Proof of concept

@shabaz123 , I am splitting this part out and assigning to self.
I think we can build the flash stgorage part and the memory protection part separately,
integrate the pieces that work, when they work.

ANALOG:INP0:RAW? to return the raw ADC value of the first ADC pin in array of analogue input pins

API to reside in src/adc
look for placeholders and TODO in scpi-def.c and adc_utils.c

If real voltage to be returned, then there is a different SCPI command for it:

ANALOG:INP0:VOLTage?
MEASure:VOLTage:DC?

    // 12-bit conversion, assume max value == ADC_VREF == 3.3 V
    const float conversion_factor = 3.3f / (1 << 12);

scpi-def is getting big. The register code should be easy to move to its own files

It would require to adapt the TinyUSB - may require to step from using TinyUSB Pico port, to using an own source base:
In the examples of TinyUSB, this is usually set in usb_descriptors.c
"123456", // 3: Serials, should use chip ID

When a non-query command is submitted, the STB is 16 (MAV still set)

DIGI:INP0? should return the status of input pin 0 (0 is the 1st entry in the index of input pins)

See DIGI:OUTP0? for inspiration
(that one queries the current status of an output pin, but it should be virtually the same)

use the develop_set_3 branch as merge target

related to the type of the setPwmAt() value parameter,
should value be 16 bits? same type as getPwmPinAt() return?

impacts:

• function setPwmAt()
• function SCPI_AnalogOutput()

What should I reset if a SCPI statement does not generate a reply?

currently I do:
queryState = 0;
bulkInStarted = false;
uint8_t status = getSTB();
status = 0;
setSTB(status);
buffer_tx_ix = 0u;
buffer_len = 0u;
rsp->USBTMC_status = USBTMC_STATUS_SUCCESS;
rsp->bmClear.BulkInFifoBytes = 0u;

The TinyUSB example doesn't have a scenario for this ...

source file structure
makefile
.gitignore

Merge branch develop-set-4 to main when feature set 4 is complete

pair with a labview driver release

currently, pins are named pins. Soon there will be input pins and maybe non-gpio types. Make it clear what the output pin variables are by adding OUT to their names.

use the feature_set_2 branch as merge target

enhance analog input HIRES module to also support differential inputs and configurable gain

TinyUSB implemented a portable way to use Pico Unique ID as USB serial number.
Once release after 1.5 is available, undo our own implement and adapt usb_descriptors.c:

remove:
#include "pico/unique_id.h"

struct

// String Descriptor Index
enum {
  STRID_LANGID = 0,
  STRID_MANUFACTURER,
  STRID_PRODUCT,
  STRID_SERIAL,
};

// array of pointer to string descriptors
char const *string_desc_arr[] =
{
  (const char[]) { 0x09, 0x04 }, // 0: is supported language is English (0x0409)
  "TinyUSB",                     // 1: Manufacturer
  "TinyUSB Device",              // 2: Product
  NULL,                          // 3: Serials will use unique ID if possible
  "TinyUSB USBTMC",              // 4: USBTMC
};

implement

uint16_t const *tud_descriptor_string_cb(uint8_t index, uint16_t langid) {
  (void) langid;
  size_t chr_count;

  switch ( index ) {
    case STRID_LANGID:
      memcpy(&_desc_str[1], string_desc_arr[0], 2);
      chr_count = 1;
      break;

    case STRID_SERIAL:
      chr_count = board_usb_get_serial(_desc_str + 1, 32);
      break;

    default:
      // Note: the 0xEE index string is a Microsoft OS 1.0 Descriptors.
      // https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/microsoft-defined-usb-descriptors

      if ( !(index < sizeof(string_desc_arr) / sizeof(string_desc_arr[0])) ) return NULL;

      const char *str = string_desc_arr[index];

      // Cap at max char
      chr_count = strlen(str);
      size_t const max_count = sizeof(_desc_str) / sizeof(_desc_str[0]) - 1; // -1 for string type
      if ( chr_count > max_count ) chr_count = max_count;

      // Convert ASCII string into UTF-16
      for ( size_t i = 0; i < chr_count; i++ ) {
        _desc_str[1 + i] = str[i];
      }
      break;
  }

  // first byte is length (including header), second byte is string type
  _desc_str[0] = (uint16_t) ((TUSB_DESC_STRING << 8) | (2 * chr_count + 2));

  return _desc_str;
}

also adapt CMake file:
target_link_libraries(tinyusb_bsp INTERFACE pico_unique_id)

I expect that, to make SRQ work, I need to remove the status byte register (STB) from the USBTMC code part (it comes from the example), and make every change integrate with the SCPI-LIB

hold until e14 challenge finished. Because the driver examples are in quiz 10, and they will change once the current feature set is released.