Code Monkey home page Code Monkey logo

tj-python's Introduction

Differences in this library compared to others

  • We don't use conflicting pins. We use software PWM for the servo and we use the SPI buss for the LED instead of using the 2 default PWM ports that are used for left and right audio.
  • We use analog audio to a speaker and use a different microphone than the small micro one everyone uses. It has significantly better pickup for the cost difference.

alt text

THINGS THAT HAVE TO BE PERFORMED ON A NEW PI

  1. Download the newest Raspbian image from: https://www.raspberrypi.org/downloads/raspbian/

    • We currently use RASPBIAN JESSIE WITH DESKTOP 2017-07-05
  2. Load the downloaded .img file onto your SD card. Raspberry Pi recommends that you use Etcher. See their Installing Operating Systems Images page: https://www.raspberrypi.org/documentation/installation/installing-images/README.md

  3. Configure the Raspberry Pi

    • Raspberry Icon --> Preferences --> Raspberry Pi Configuration
      • System:
        • Underscan: Disabled
      • Interfaces:
        • Camera: Enabled
        • SSH: Enabled
        • Serial: Enabled
      • Performance:
        • GPU Memory: 256
      • Localisation:
        • Locale: click Set Locale...
          • Language: en (English)
          • Country: US (USA)
        • Timezone: click Set Timezone...
          • Area: US
          • Location: Central
        • Keyboard: click Set Keyboard...
          • Country: United States
          • Variant: English (US) scroll up to see it at the top
    • Then reboot the pi sudo reboot or use the menu
  4. Set the default audio to be analog audio. sudo raspi-config

    • Pick option 7 Advanced Options
    • Pick option A4 Audio
    • Pick option 1 Force 3.5mm ('headphone') jack
    • Press the Enter Key
    • Use the Left and Right arrow keys to select <Finish> and press the Enter Keyboard

Then you can run the script located in config/setup.sh or you can execute the commands manually below.

  1. Update the dist list: sudo apt-get update
  2. Upgrade the apps sudo apt-get upgrade
    • Tell is yes if it asks if you want to continue Y
  3. Upgrade the distro sudo apt-get dist-upgrade
  4. Install vim sudo apt-get install vim
    • Vim config TODO
    • Git config use vim TODO
  5. REMOVE Install screen sudo apt-get install screen
  6. Install Python 3 and set as default
    • Already default
  7. Install system dependencies they are found on the pyaudio page(Below)
    • sudo apt-get install portaudio19-dev
    • sudo apt-get install python-all-dev
    • sudo apt-get install python3-dev
    • sudo apt-get install python3-pyaudio python-pyaudio !!!THIS DOES NOT WORK ON THE RPi!!!
    • TODO: Need to add the special code from the non standard dist that makes PyAudio work. This may not be correct if the new library doesn't require it!
  8. Install system dependencies for the watson-developer-cloud
    • sudo apt-get install libssl-dev libffi-dev build-essential scons swig
    • sudo easy_install3 --upgrade watson-developer-cloud
  9. Install the adafruit stuff
    • sudo pip3 install RPi.GPIO
  10. Install this for the LED controls
    • git clone https://github.com/jgarff/rpi_ws281x.git
    • cd rpi_ws281x
    • scons
    • cd python
    • sudo python3 setup.py install
  11. You need to copy the asound.conf file to /etc/
    • sudo cp asound.conf /etc/asound.conf
  12. You need to copy the asound file to ~/.asound
    • cp .asoundrc ~/.asoundrc
  13. Install VLC to play the music
    • sudo apt-get install vlc
  14. Install the python libraries required.
    • sudo easy_install3 websocket-client
    • sudo pip3 install colour
    • sudo pip3 install python-vlc

Thinking about adding: sudo pip3 install SpeechRecognition

IBM's TJBot code rewritten in Python.

NEEDS TO BE UPDATED

This is initial infromation that somewhat applies to the code in this repository but it is not up to date at this time.

  • Things we want to be able to do in python:
  • Speech to text, preferably more live than waiting for no sound to process, but whatever
  • Again need to figure out how to add some keywords to his vocabulary
  • Function waitforsomething(resetafter)
  • Maybe if it keeps hearing something after a while it says idk how to help
  • Text to speech, make sure you can do the character inflection and verbal tone stuff with taht one female voice.
    • Function speak(text) maybe speak(text, emotion)
  • Conversation, send text to the conversation module and have it return a string, possibly with inflection or some kind of string at the beginning [wave] where [] denotes a command and the inside is a command that TJ bot will process and do and then remove it before it speaks it or something.
    • Function sendconversation(conversation), will wait for a response
  • Tone analysis, send the text from speech to text through the tone to add some context for the conversation part. Figure out how to use the context stuff in teh conversation stuff.
    • Function analyzetone(text), returns the emotion array
    • Function analyzetonesimple(text), returns the highest option. Ex “sad”
  • This Personality thing, no idea what it is.

Servo

Function wave(duration) or wave(count)
Function setangle(angle) or setangle(angle, speed)
Function up(), down(), forward() ---> just sends an angle to the set angle function
Function stop()

LED

Function setcolor(r,g,b) or setcolor(hex) or setcolor(color)
Function playsequence(sequence)
Sequence is a list of actions
A sequence has a repeat infinite, repeat count options
An action for the LED has a color and a duration
Function rainbow(speed) → does stuff with colors
Function strobe(speed) → strobes white at the speed
Function stop()

Music

Function playmusic(sound file)
Function volup(), voldown(), volmute()
Function pause(), stop(), restart()

Camera

Take a picture and just show it on the screen
Show a live video stream

tj-python's People

Contributors

bmcginn1 avatar brandon-gong avatar datcomputerguy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

tj-python's Issues

Conversation Command Filtering

Need to work on the syntax that pulls commands from the returned conversation text. Currently using ~ACTION but I am thinking we need to do something with the tag syntax
<tj action="LED">
or
<tj action="LED" color="RED"><tj action="SPEAK">TURN LIGHT ON, SPEAK THIS, AND THEN TURN LIGHT OFF></tj></tj>

That way actions can be triggered in the code that last a duration but the programmer doesn't actually have to know the duration of the inner action. And if another action is called it always trumps, basically how we have the treads working right now.

Killing the program

Right now we get zombie processes that seem to be a result of all of the different threads and processes that are being run. We need to program in an interpreter for keys like Ctrl+C and Ctrl+X so that they can actually kill the entire program and then there will be no need for the script that kills all of the existing python processes before running it again.

Audio controls that conflict with GUI

Need to work on the audio controls in raspbian and figure out how to disable the GUI from overwriting the system changes. Also need to add the configuration settings to use the analog audio out as the default and use the usb mic as the default input.

Command line window

Something that could be nice is to have the ability to enter commands wether they be friendly text or literal python code into the code window while it is running the program. This way we could show the kids something like led on and the led turn on. Or we could do ledP.setColor("red") or whatever we wanted. Basically create an interactive console.

This could also combine the use of some other terminal techniques I have seen that allow for the screen to be divided into multiple "windows" so you could have the program running at the top and then have a command area below. This is advanced I know but I know people have taken the time to program some frameworks that allow us to do this.

PROMPT AREA

DEBUG/OUTPUT AREA
COMMAND ENTER AREA
STATUS BAR AT BOTTOM

This is just a thought that we can explore.
The first version of adding this feature would be to just have 3 threads that output text onto a screen and then use screen to view them simultaneously and then later on it is just combining the output from the 3 different threads or processes or whatever that we end up using to do this.

PyDoc Lint

Need to document all of the code that conforms with the lint standards and so it works well with pydoc

Need to add LICENSE

need to add a license file with the rules or whatever for this repository

streaming.py improvements

Couple of fixes:

  • Maybe add debug levels. Right now bunches of scary alsa messages are getting output into console, and somewhere hidden in there is what the user actually says. A better user interface would be just to print what the user says while streaming transcription. But also be able to re-enable those messages for debug purposes.
  • Automatically catch ErrNo 9981: Input Overflowed errors. It seems like most of the kids except like one or two got this error, and we fixed all of the issues by simply changing the RATE from 48000 to 44100. How hard can it be to automatically catch the error and retry with a different RATE?
  • Don't have a hardcoded record time. Right now, streaming.py records for 3 seconds and then stops. So the kids would have to squeeze whatever they have to say into three seconds. streaming.py needs to be able to record until the kid is done talking. This may be harder, because it's streaming. I'll see. I can't simply take the code from speechRecorder.py because the way speechRecorder.py works is it stores all of the audio data in these massive deques and then writes the entire deque to a file once its done. Not very streaming-ish.
  • Figure out how to use custom names. Some of the kids had to change their robot's names to common words like "table" because streaming transcription did not support custom terms. Information about customization is found here.

Create setup script

Need to create a setup script that works on the Pi and installs all of the dependencies required.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.