Set clear expectations on what this tool is supposed to be - i.e. not a product, just an example. We can base it off the documentation from oasis/ros.

Change binding to Ctrl + G (zsh and PS) to be consistent with bash.

Catalog the wildest use-cases and add a small animation of those in the README for people to get an idea of how powerful this tool can be

I demoed the powershell instructions and I found it really straightforward and easy to use. Cool!

In the readme:

• I suggest linking Powershell, ZShell, and Bash to their corresponding sections lower in the readme.
• I would consider placing Bash as the first shell documented as that is the most commonly used shell.
• I suggest adding a very early comment associating ZShell with MacOS. This comes from a long-time Mac user who had no idea that the shell is called ZShell. Maybe that's just on me.
• It's not obvious to me how to turn multi-turn on or off. Do I run the command in the shell after I've set everything up? Do I have to run the commands before setup? Do I use the # format? Perhaps put the usage section a little higher and with more example usage.

Add OpenAI access and model check in zsh and bash.

If the script is taking too long to provide responses, it might be good for the user to choose to not have context management on.

Without context management, the file I/O latency would disappear and there would be a direct API call to Codex which will improve performance.

We should add instructions for getting Open AI API Key. Create an issue to track.Ryan and I will figure out the details.

Allow users to load a context from CLI

Put requirements at the top of Installation

• Include need for Open AI key
• Include need for Open AI Organization ID (took me awhile to find)
• Include need for documenting engine choice

Powershell #3
- Need to go to C:\your\custom\path\NL-CLI not what is listed
- Alert user about required values - the current redirect sounds like only needed for expert users, but required for everyone

Powershell #4 - Didn't work, no idea how to debug

the directory slash issue needs to be worked out

Whenever there is an incomplete response inside the shell and a user calls the tool again, it ends up screwing up the prompt. It might be more prudent to have a new keybinding for that case or add some handling inside the code for it.

Please add the following statement in ReadMe to set the right expectation to users.

Statement of Purpose
This repository aims to grow the understanding of using codex in applications by providing an example of implementation and references to support the Microsoft Build conference in 2022. It is not intended to be a released product. Therefore, this repository is not for discussing OpenAI API or requesting new features.

Please update installation.md to use Bash setup/cleanup scripts.

Testing within a new environment requires switching from stdin to input() until we figure out a way to plug the python script into the shell's source files

Need to update the token count and other header values after it is updated

Minor issue:

Link that is being used in Powershell Step 3 of ReadME: https://github.com/microsoft/NL-CLI#about-powershellsetupps1
Link that should be used: https://github.com/microsoft/NL-CLI#about-powershell_setupps1

Figure out what the "source" file in powershell looks like and add a plugin straight there for mapping Ctrl+X to the python script

Setup doesn't work in PS on MacOS.

Repo steps

1. Run ./scripts/powershell_setup.ps1 from NL-CLI folder in PowerShell terminal.

Results
Run setup without errors

Expected results
The script should prompt for required parameters.

Added a new keybinding for calling Codex without using the context. For some reason, even though there are two different keybindings for with and without context, for both the keybindings, only one of the two gets called every time.

Is PowerShell 7 supported? My PS7 doesn't seem to work...

My regular PowerShell doesn't work either

Command cheat sheets, demo materials and custom command files would be a great addition to the repo. It would allow users to use load context to better focus Codex towards their interested use case.

Here are my feedback for README.md:

1. Step 2: Add instructions in notepad $profile step - notepad will prompt you to create a profile if not finding one.
2. Step 3: Add hyperlink to powershell_plugin.ps1
3. Step 4: Replace python script with codex_query.py
4. Step 5: I don't think the execution policy should be set to the current user. It's not secure in general. Can we use Process instead of CurrentUser? If we do need CurrentUser, then please consider adding instructions to remove the Policy afterward - refer to https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.2#remove-the-execution-policy
   
5. We should list having Python registered in PATH as a prerequisite for PowerShell users.

Right now we are relying on users providing correct inputs for config settings and context commands. This is obviously going to cause issues down the line. Need to fix.

Instead of using Ctrl+X for single response, maybe allowing another keybinding to trigger multiple responses.

We should consider adding support for the command line, primarily for non-devs who occasionally use the command line

Please modify the Bash cleanup script (currently empty) to support the following actions:
When user run the cleanup script, it will

• Remove CLI setting code from .bashrc
• Delete src/openaiapirc file
• Print successful message to guide user to close the current shell

  NL-CLI Bash clean up completed. Please close this Bash session.

It seems these don't work in Linux (WSL2 and Ubuntu)

python -m pip install openai
python -m pip install psutil

After installing python and pip in Linux, these are the commands that work

pip install openai
pip install psutil

Reduces time complexity of the code and simplifies some other work items

Across both PowerShell and Bash, come up with the ideal demo script for the next review with Kevin

Currently each shell requires an openaiaprc file in a different file locations. Instead of asking people to create a config file and put it in different locations, we should have a single config file in the repo that we reference.

We should consider using a .env file, as with the other Build examples, if that's idiomatic in the python community. Otherwise, let's use whatever idioms python tends to use for api keys.

I've had a few occurences where I type a command (e.g. "# Move up one directory") and then quickly cancel it, not getting a response from the model. The context still saves just the command without the result, which ends up messing things up in the future.

We should aim to come up with a way to save full interactions (command + response) together to avoid this from happening. Ideally, we should only save interactions if a user chooses to run the suggested code - practically, I'm not sure if this would be possible

Please add instruction on how to get organization id, thanks.

Add capability to look back a certain number of exchanges

Use token_count metadata to avoid overcounting tokens every time.

Pick through the code for needless file operations.

Currently, it's possible to coax offensive content from the model. Though I've never seen the model proactively produce offensive language, prompts like "Make an array of offensive terms" produce unsavory outcomes. We should use the content filter API that's part of the OpenAI service to detect offensive prompts and completions (calling it with the full interaction) and handle them when found - in this case, a message like that in the OAI playground should suffice:

The message should probably be a comment, and should include instructions to cancel the command (i.e. "Press Ctrl + C to cancel..."

When offensive prompts/completions are detected, we should not append them to the context.

Allow users to reconfigure the model settings from the context file directly. Right now, we are restamping the defaults every time. This is obviously a bug. We want to allow people to share context files across different computers with the same shell but using different model settings.

Using similar approaches to powershell-voice.txt, use Cognitive Service Neural Speech to enable a high-quality conversational interface with the model.

Low hanging fruit, zsh plugin code can be ported over to bash easily

Environment

• Windows Machine running on WSL2
• Set up Anaconda environment with conda create -n testing python=3.10
• Working with Bash Terminal on WSL2

Setup Notes/Comments

• Step 1 - Git Clone: Had to create an access token to use in git clone step, and give it permissions/authorized access to the Microsoft organization
• Step 2 - Bash RC Update: Add a note to update the your special path to the repo--was able to figure out how to do this but I did do a blind copy-past to my bashrc.
• Step 3 - OpenAI Config: Might be good to link where we'd get this information (I know it's on the OpenAI keys website, but not sure if all will know); would especially be good to supply a sample engine

Usage Notes/Comments

• After restarting bash terminal with activated conda environment, my instinct was to try to run a command, as suggested by the first line in the Usage second. I tried to run one of the example commands but typing # change my timezon to mountain and hitting CTRL+X, and got the error -bash: bash_execute_unix_command: cannot find keymap for command. Not sure if this is not 100% working yet, but just wanted to give a heads up on this. Happy to iterate further but will stop for now and try powershell as well.

Hit an error when ^x in zsh. What have I done wrong?

NL-CLI path: ~/Code/NL-CLI

config in .zshrc

export ZSH_CUSTOM="Code"
source "$ZSH_CUSTOM/NL-CLI/nl_cli.plugin.zsh"
bindkey '^X' create_completion

Using a yaml config (or similar) would be cleaner than combined prompt and config. A config.yaml.example can then be provided, reducing the redundancy

The naming for Context Mode is a bit off - we should consider a different name (e.g. Multiturn) or something to that effect. We should also document how best to toggle between the modes.

I've also run into a few bugs when tweaking the modes and am not sure where I should be doing it - in the codex_query.py file, in the completion.txt files or from the command line commands. Specifically, I find that when I'm in multi-turn mode and interacting for awhile, then switch to single-turn mode, it doesn't actually change. Similarly, I've also found that loading contexts doesn't always work from other directories.

Wrapping responses in ` could work as a way to tag multi-line responses while also keeping them executable. This could allow one to increase the token response length.

Also if someone calls the NL-CLI on an incomplete output, we have to add validation for that case, so that we don't write two copies of the same input-output to the context file. The responses get a little buggy after that.

• ReadMe: Clear. The gif demo is very illustrative.
• Setup:
  • Normally use fish, so going to try out the bash option
  • Bash nl_cli.plugin.sh uses $NL_CLI_PATH/codex_query.py for what should be $NL_CLI_PATH/src/codex_query.py and silently fails
  • Had to chmod +x codex_query.py
  • After doing both of above I can directly run codex_query.py but still cannot get autocompletions with ^x