Speech:Spring 2018 Software Group


 * Home
 * Semesters
 * Spring 2018
 * Proposal
 * Report
 * Information - General Project Information
 * Experiments - List of speech experiments

Groups

 * Systems Group
 * Experiment Group
 * Data Group
 * Modeling Group
 * [Software Group]

Group Member Logs

 * Wesley Couturier
 * Faruk Durakovic
 * Danielle LeBoeuf
 * Lamia Mukanovic
 * Joshua Young

Overall Decode Notes
sphinx3/src/libs3decoder/libsearch/srch_time_switch_tree.c

Jonas wanted Wes to look into finding out what pl->pheurtype resolves to. We know that it's one of four options:
 * 0 - Bypass : which simply means that it is ignored
 * 1 - Sum of Max
 * 2 - Sum of Average
 * 3 - Sum of 1st Senome

when delving into the file, we see that the function where we can see pl->pheurtype is referenced into srch_TST_funcs which is a structure type that contains pointer references for functions. The function is called in other files, but it is not set there.

When greping for references to pheurtype, there's a reference within the top level include/ directory, within the file cmdln_macro.h. The specific lines are: { "-pheurtype", \ ARG_INT32, \ "0", \     "0 = bypass, 1= sum of max, 2 = sum of avg, 3 = sum of 1st senones only" \ } It turns out that pheurtype is a flag that needs to be passed in when we decode, otherwise it defaults to a bypass. However, none of our sphinx3 decode runs have passed in the flag manually, and it's not contained as a separate parameter within the sphinx_decode.cfg file as well within the etc/ directory within our experiments.

Tasks

 * check CMU sphinx 3 main files and compare to ours
 * - if that doesn’t work look up other sphinx 3 main files and compare to ours
 * find documentation on the individual files themselves
 * all go through main files- pick which ones you think are the most important main file and describe why
 * - Main file directory: /mnt/main/root/sphinx3/src/programs

Source Code Pages

Color Codes
 *  Danielle is purple 
 *  Lamia is red 
 *  Faruk is green 
 *  Josh is orange 
 *  Wesley is blue 

Useful link: http://www.cs.jhu.edu/~juri/pdf/mllr-rwth-2005.pdf about MLLR, and HMM

Overview

We are the first generation that will look at the specific decoding of the files in the decoder. Going through the files these are the terms that we have found helpful in searching about the main_decode.c file.

Cepstra- A type of file that has analyzed human speech; quefrency analysis

Gaussian- Represents the probability density function of a normally distributed random variable

Senone- A sound detector. A variety of sound detectors can be represented by a small amount of distinct short sound detectors.

Intellisense- A list of features within a code debugger that makes it easier for code to be identified, found, explored, and edited.

DAG search- A directed graph that has a topological ordering, a sequence of the vertices such that every edge is directed from earlier to later in the sequence.

With the responsibility of understanding how the Decoder works, the Software Team will contribute extensive documentation across multiple files and; time permitting, a recompile of the speech Decoder. Initially the team will start analyzing main_decode.c to ensure the quality of the decode and then we will each take a different file as the semester progresses. This will improve the opportunity for other Teams to make distinct amenities for more accurate voice recognition.

Furthermore, we will collectively do extensive research on each of the "main" files and break down what each piece of code does and how it is relevant to the overall project. So far, we have found files in languages C/C++, Python, and Perl. We have found that this project is best explored in Visual Studio (not Visual Studio Code), although a debugger with IntelliSense should suffice. We find it best to create a visual map, located in the back of the server room, to display how each "main" file is connected and how they work off of each other.

Objectives The Software Team is tasked with documenting the functions of the files related to the Sphinx 3 Decoder. Starting with main_decode.c where the MAIN function in located, we will branch out through the files connected to the Main file to map how the files connect together. We will look into and setup SVN version control to allow Sphinx files to be edited. If time permits, we would also like to refactor the Sphinx 3 Decoder to add any updates to improve accuracy. We will also create a Wiki page for documentation for us the next Software Team to contribute to.

Timeline


 * Decode Sphinx Decoder
 * Start decoding the main_decode.c because that is where the main function is located in (Team) 2/13 - 2/20
 * Comment each line of the source code and add overall conclusion of what the file does (Team) 2/13 - 2/20
 * Look at corpus.c to determine how the functions relate to main_decode.c (Team) 2/20 - 2/26
 * Look at sphinx3_decode.c to determine the connection to main_decode.c and why it searches through that file (Team) 2/20 - 2/26
 * Continue looking through different files that are connected to the main_decode.c and how they impact the overall decoder (Team) 2/20 - 5/8
 * Setup File Version Control
 * Look into Revision Control System (RCS) and Subversion (SVN) (Wesley, Danielle) 2/20 - 2/28
 * Determine which one is best suitable for Sphinx3 and set up files in version control system (Wesley, Danielle) 2/20 - 2/28
 * Set up desired version control system on Majestix, because we will be doing software updates on this drone (Wesley, Danielle) 2/28 - 3/7
 * Confirm it is compatible with Sphinx3 and overall Speech Recognition Software - (Wesley, Danielle) 2/28 - 3/7
 * Test to see if the version control system is working on Majestix, and record any problems it encounters - (Wesley, Danielle) 2/28 - 3/7
 * Recompile / Update System Programs
 * Run an experiment to get a baseline before any updates/recompiles (Josh, Lamia) 3/7 - 3/13
 * Disconnect one drone from Caesar. This drone will most likely be Majestix, because it has GCC already installed (Josh, Lamia) 3/7 - 3/13
 * Create files with system folder names to compare after the updates. To verify what has been changed and/or updated (Josh, Lamia) 3/7 - 3/13
 * Recompile software including the Decoder (Sphinx3) (Josh, Lamia) 3/9 - 3/18
 * Compare the current system file folders with the originals to verify changes (Josh, Lamia) 3/9 - 3/18
 * Reconnect drone to Caesar (Josh, Lamia) 3/9 - 3/18
 * Run Identical experiment to make sure updates did not adversely affect results (Josh, Lamia) 3/9 - 3/18
 * Create Physical Mapping of Paths used in Sphinx Decoder
 * Take Decoded Sphinx Decoder and the various files used and map them on a wall for visualization (Team) 2/20 - 5/8
 * Use visual mapping to create an easy to understand model for the other teams (Faruk) 2/20 - 5/8
 * Use visual model in conjunction with other teams to facilitate understanding of the rest of the sphinx decoder with all individuals and to review other useful features (Team) 2/20 - 5/8

What RCS Is
Revision Control System (RCS) is a command-line based, version control system that was created for, and is to be used by a UNIX based OS. It is currently maintained by the GNU Project under an open and free license. It is to be used similarly how one may rent a car from an agency. First, the agency checks-in a car, then leases or rents it out to people who want to use it. This is done by "checking-out" the file, making your edits, then checking-in the file once you're done so others can make edits. When you're ready to compile the files that you're working on (using make or cmake), we can specify the working directory containing all files that RCS has checked-in, and make will auto-compile the changes for you. See usage below for additional tips on how to use it effectively.

How We Installed It (General Installation Instructions)
This set of instructions will be very similar to the system's groups own instructions for installing software Here

We first want to get the O.K. from Jonas of
 * 1) What we are installing and
 * 2) Where we are installing it.

This is probably the easiest step (at this time). The next few steps will eventually require you to be physically present at the server room, right next to room P149, and may take up to 3 hours your first time doing it. You will need an NFC key pass to get in.

After getting the okay, we want to isolate the drone we're working on.This is done by unmounting it from Caesar.
 * Run the command umount -a or umount /mnt/main.

The latter of the two is more specific, although it could be the preferred method as there could be other mounts we want to ignore.
 * We double check that it worked by looking in the mount directory /mnt/ to see if main contains anything. If it's unmounted, it will contain nothing, and will simply seem like any other sub-directory.

Next, and this is where it WILL require you to be in the server room, we need to physically connect to the outside world, and disconnect from everywhere else. At this time, we're doing this physically and it may change in the future.
 * Remove all Ethernet cables from the back of the drone that you're working on.
 * Then connect One cable to one of the ethernet ports on the drone, and connect it's other end to an open port on the lower levels of the stack next to the drones.

For our class, the port was on the lowest level. A specified port may be given in future classes.
 * Once done, we need to make sure that we can connect to google. Go to the default working station on top of the drones, open a terminal, and type in ping -c 3 8.8.8.8. This is to ping Google.com's DNS server 3 times. If it performs successfully, then you're good. If not, then try a different ethernet port on your drone. You have have to fenagle a little bit because of firewall issues. This may be updated later on.

Once connected to the internet, we need to log certain directory file contents before the install in order to see the changes. We do this in case we need to see what files were possibility affected. The specified file directories are below, and are formated such that you can copy and paste the entire command from whatever directory you're in to save your results. This may be turned into a simple bash script later:

ls -al /bin/ > bin.txt ls -al /lib/ > lib.txt ls -al /sbin/ > sbin.txt ls -al /sys/ > sys.txt ls -al /usr/bin/ > usr-bin.txt ls -al /usr/libexec/ > usr-libexec.txt ls -al /usr/lib/ > usr-lib.txt ls -al /usr/local/bin/ > usr-local-bin.txt ls -al /usr/local/lib/ > usr-local-lib.txt ls -al /usr/sbin/ > usr-sbin.txt ls -al /usr/share/libgnomekbd/ > usr-share-libgnomekbd.txt ls -al /usr/share/libgweather/ > usr-share-libgweather.txt ls -al /usr/share/libhangul/ > usr-share-libhangul.txt ls -al /usr/share/librarian/ > usr-share-librarian.txt ls -al /usr/share/libthai/ > usr-share-libthai.txt ls -al /usr/share/libtool/ > usr-share-libtool.txt ls -al /usr/share/libwacom/ > usr-share-libwacom.txt ls -al /usr/ > usr.txt ls -al /var/lib/ > var-lib.txt ls -al /var/local/ > var-local.txt

Once you've documented your directories, it's finally time to install the software.
 * With RHEL, we use the yum command to do everything. The specific command for RCS was yum rcs install. It will most likely prompt you confirm the install, this is a good double check to make sure we aren't installing something we don't know about. For any other install, you may need to add a repository for it to find the software you want.

Let the installer do it's job, and once it's done we need to then take another snapshot of our working directories. Use the above step to do so again, but in another, nicely labeled directory. This may need to be used to find out if anything went wrong for future use.

After that, simply disconnect your drone from the outside world and reconnect everything you had before.
 * Lastly, remount Caesar by typing the command mount -a.

RCS Usage
... to be filled out later ...

Steps needed to install Sphinx 3 Decoder on Rome (recompile)

 * 1. Go to Sphinx Download Page and browse the folders.
 * 2. Find - Older releases and files could be found on  SourceForge
 * 3. Since we still running Sphinx3, navigate to the Sphinx3 folder, then click 0.8, and lastly click sphinx3-0.8.tar.gz. This is the final page you should end up on Click to download for Sphinx3-0.8
 * 4. Once you click sphinx3-0.8.tar.gz it will download, I suggest placing it in a folder on your Desktop.
 * 5. Since Rome does not have same directories as Caesar, I had to make a folder on /mnt/main/ called sphinx3, therefore on Rome and Caesar I had a directory called /mnt/main/sphinx3
 * 6. On Rome, under /root/sphinx, Professor Jonas created an empty folder called sphinx for the recompile.
 * If there isn't folder you can create one using mkdir
 * 7. I have a Mac, and I used FileZilla for sftp program to put it on the desired server (i.e Rome)
 * 8. On the left hand side of FileZilla is your local machine, navigate to where the download of sphinx3-0.8.tar.gz is (should be in a folder on your Desktop)
 * 9. On the right hand side, navigate to the common directory of mnt/main/sphinx3.
 * 10. Drag the sphinx3-0.8.tar.gz from the left hand side to the right hand side to put it on the server.
 * 11. On the terminal, navigate to /mnt/main/sphinx3 on Rome
 * 12. Once you see sphinx3-0.8.tar.gz in /mnt/main/sphinx3 you can now move it to /root/sphinx
 * 13. To move it use the following command sudo mv frompath topath
 * In my case it would be sudo mv /mnt/main/sphinx3 /root/sphinx/sphinx3
 * It will already create the folder sphinx3 there is no need to create a folder called sphinx3 on /root/sphinx
 * 14. Unzip the sphinx3-0.8.tar.gz using the command
 * tar -xvzf
 * In my case it was tar -xvzf sphinx3-0.8.tar.gz
 * 15. A bunch of files will extract in /root/sphinx/sphinx3
 * 16. Run the command sphinx3-0.8/configure or ./configure
 * Be careful there are other configure files make sure it just says ./configure (no extension after)
 * 17. Per Professor Jonas's instructions I had to change the Makefile
 * 18. After running ./configure, if you ls into /root/sphinx/sphinx3 you will see a list of 12 folders
 * config.log | doc | libtool | model |  sphinx3-0.8(unzipped Sphinx3 decoder)  |  sphinx3.pc  | config.status|  include | Makefile | scripts| sphinx3-0.8.tar.gz (zipped Sphinx3 Decoder) | src
 * 18. In order to edit the Makefile
 * Use nano (for some reason emacs doesn't work on Rome)
 * nano
 * ctrl + W (searches)
 * current bin directory /usr/local/bin
 * I changed it to root/sphinx/sphinx3
 * ctrl + O, saves the changes, hit YES to save the edited file
 * ctrl + X to exit