Speech:Spring 2016 Nigel Swanson Log


 * Home
 * Semesters
 * Spring 2016
 * Proposal
 * Report
 * Information - General Project Information
 * Experiments - List of speech experiments

Week Ending February 9, 2016
2/3/16 - Log into caesar, get familiar with the system and data, start identifying tasks to do for the group.
 * Task:

2/7/16 - Read logs from members of the Tools group from last year.

2/8/16 - Read logs from the current week.

2/9/16 - Remotely log into caesar, setup user on caesar, learn perl scripting, determine old scripts that have not been used in a long time, read through the running a train process, install Ubuntu server on virtualbox, install emacs on local Ubuntu server, start learning emacs.


 * Results:

2/3/16 - Was able to log into caesar via ssh. Was able to get more familiar with the system, specifically with the perl scripts. Started to identify old scripts (usually identified by #) and also identified an entire folder named 'DELETE' full of old scripts. Also found that the wiki page does not mention all of the scripts that could still be of use.

2/7/16 - Read logs from members of the Tools group from last year.

2/8/16 - Read logs from the current week.

2/9/16 - Was able to remotely log in to caesar after setting up a connection to UNH. I then changed the password of my user account so I could then use my account on caesar. I went through two tutorials on perl scripting so I could learn the syntax and better understand the perl scripts already created. I found that files do not have a last executed time, they only have a last accessed time. This means that even if a file was read the time will be updated. So determining unused scripts by their last execution time will not be possible. I read through the steps involved with running and decoding the train. I installed an Ubuntu server on virtualbox for local testing of potential software to be installed on caesar. I installed emacs on my local Ubuntu server and started using emacs to learn more about it.


 * Plan:

2/3/16 - Learn more about software for unix such as emacs, run a train to learn how the process is done, look through scripts and specifically identify duplicate or outdated ones to delete.

2/7/16 - Continue reading logs.

2/8/16 - Continue reading logs.

2/9/16 - Continue testing and installing unix software on my local Ubuntu server to see if it should be installed on caesar.


 * Concerns:

Unable to determine old perl files based on last execution time.

Week Ending February 16, 2016

 * Task:

2/10/16 - Identify tasks to put into the proposal. Work on learning experiment process.

2/13/16 - Read logs from the current week.

2/15/16 - Read logs.

2/17/16 - Run experiments to learn the process.


 * Results:

2/10/16 - Talked with group about tasks to put into the proposal. Started looking into the experiment process.

2/13/16 - Read logs from the current week.

2/15/16 - Read logs.

2/17/16 - Created directory for the Tools group 0286. The first experiment that I ran I did not create a sub directory for the experiment. Running the train resulted in a failure which I believe happened because I did not create a sub directory for the experiment. On my second attempt of running the train I tried the createWiki_Sub_Experiment.pl but it gave a bunch of errors. So I manually created a sub directory. Before running step 6 from the Run train setup page I tried createWiki_Sub_Experiment.pl again and was successful after putting the 'perl' command in front of the script name. Running the train was unsuccessful and failed at the same spot as the previous attempt. There error was, Can not use models created by Sphinx-II. I believe this may have been because I did not run the createWiki_Sub_Experiment.pl at the beginning of the process, but will continue to try to solve this issue. I also identified a potential way to setup a redhat environment before redhat is set up on the new servers. Amazon Web Services offers redhat as an instance so I will be investigating to see if this will work for our testing needs.

2/10/16 - To run a successful experiment.
 * Plan:

2/13/16 - Continue reading logs.

2/15/16 - Continue reading logs.

2/17/16 - Running a successful train, getting redhat on Amazon Web Services running.
 * Concerns:

Being able to run a train successfully. Seeing if redhat through Amazon Web Services will work.

Week Ending February 23, 2016

 * Task:

2/20/16 - Reading current logs from the week.

2/21/16 - Research new versions of SphinxTrain and CMU Dictionary. Meet with group about proposal/research.

2/22/16 - Read logs.

2/23/16 - Continue research and documentation of SphinxTrain and CMU Dictionary.


 * Results:

2/20/16 - Read current logs from the week.

2/21/16 - Researched SphinxTrain and found version 1.0.8. Found out that they researched this last year but they didn't put this information in the Speech Software Functionality Wiki, so I updated the SphinxTrain section. I then did research on the CMU Dictionary and found out that there is not a new version since last year. I updated the link in the Speech Software Functionality Wiki for the newest version to the correct location. I also met with my group to talk about our research and about finalizing the proposal.

2/22/16 - Read logs.

2/23/16 - Found information on the latest versions of SphinxTrain and CMU Dictionary. Updated part of the proposal.


 * Plan:

2/20/16 - Read logs.

2/21/16 - More research on software tools and updating part of the proposal.

2/22/16 - Read logs.

2/23/16 - Let others know about my findings on SphinxTrain and CMU Dictionary.


 * Concerns:

None

Week Ending March 1, 2016

 * Task:

2/27/16 - Read logs from the week.

2/28/16 - Meet with group about final edits in proposal.

2/29/16 - Read logs.

3/1/16 - Find any other sources related to CMU Dictionary or Sphinx Trainer. Read through Tom's log on the sox fix. Read up on tree linux package.


 * Results:

2/27/16 - Read logs from the week.

2/28/16 - Met with group about the final edits in the proposal.

2/29/16 - Read logs.

3/1/16 - Didn't find any other sources related to CMU Dictionary or Sphinx Trainer, probably since there is not a new version of the software, still have same recommendation. Read through Tom's log on the steps he took to get sox working. Read up on the tree linux package.


 * Plan:

2/27/16 - Continue reading logs.

2/28/16 - Work on tasks from the proposal.

2/29/16 - Continue reading logs.
 * Concerns:

None

Week Ending March 8, 2016

 * Task:

3/2/16 - Be able to access sandbox server and start planning when to install software. Additions to wiki.

3/5/16 - Read logs.

3/7/16 - Read logs.

3/8/16 - Add information and recommend or not the update for Sphinx Trainer and CMU Dictionary.


 * Results:

3/2/16 - Was able to log onto sandbox server where the tools group is going to be installing and testing software for recommendation for Caesar. Talked with group about what list of directories we should backup, and also discussed alternative solutions rather than using tape drives for backup. Also found that google code, which we were using for some backups, was shut down earlier in 2016. Google does offer an alternative solution but we may not even need a cloud backup if we use our implementation for rsync backup. Planned updates and planned consolidation of findings of software versions onto one page.

3/5/16 - Read logs from the week.

3/7/16 - Read logs.

3/8/16 - Added my recommendation of Sphinx Trainer and CMU Dictionary to the tools group page.


 * Plan:

3/2/16 - Update wiki.

3/5/16 - Continue reading logs.

3/7/16 - Continue reading logs.

None
 * Concerns:

Week Ending March 22, 2016
3/19/16 - Research to better understand train and decode process.
 * Task:

3/20/16 - Read logs.

3/21/16 - More research on train and decode process.

3/22/16 - Run a successful train and decode.


 * Results:

3/19/16 - Learned about the process of setting up sphinx from scratch and learned what the scripts do.

3/20/16 - Read logs.

3/21/16 - Read more on train and decode process.

3/22/16 - I was able to ssh into obelix and get into the experiment folder. I could not get the createWiki_Sub_Experiment.pl script to successfully run. I tried it as root and my user, with and without the perl command, specific path from the current directory and it still would not run correctly. After reading through the perl code it looks like it is still using http and foss.unh.edu was updated to use https so that is most likely the issue. Going to catch up with group to see how others were able to do this or a workaround for this step of the process.


 * Plan:

3/19/16 - Learn more about train and decode process.

3/20/16 - Continue reading logs.

3/21/16 - Continue learning more about the train and decode process.

3/22/16 - Figure out issues with train and decode process.


 * Concerns:

3/22/16 - Running a successful train.

Week Ending March 29, 2016

 * Task:

3/26/16 - Read logs.

3/27/16 - Learn more about install process for majestix. Catch up with Tools group on tasks.

3/28/16 - Run a train and decode successfully.

3/29/16 - Read logs, stay caught up with group on gcc installation.


 * Results:

3/26/16 - Read logs.

3/27/16 - Caught up on install plan for the software that we are going to be installing on majestix. Researched ssh problem potential fixes. Met with Tools group and talked about and worked on the ssh issue and install plan.

3/28/16 - Started first train 286/005 which was unsuccessful. This is because the first_5hr corpus was moved into a different directory. I started a new train 286/006 and was able to start the train successfully by pointing to the new 'old/first_5hr/train' location. I found a couple of other differences in the steps. Such as certain scripts like the lm_create.pl script can just be accessed from the local directory and not the full directory. Found out that to run the decode using a train in the same sub experiment is 'nohup ./run_decode.pl 006 0286/006 1000 &' since it was not specified in the wiki. I also initially got the scoring of the decode before it was finished. I then went back and got the full scoring after the decode was finished. The final result was that I was able to successfully run a train and decode.

3/29/16 - Read logs. Stayed caught up on internet connection setup for majestix. Investigated other ways of installing a linux package offline.


 * Plan:

3/26/16 - Continue reading logs.

3/27/16 - Installing tree once gcc is installed on majestix.

3/28/16 - Continue learning more about train and decode.

3/29/16 - Continue reading logs.


 * Concerns:

3/29/16 - Installing tree after gcc is installed.

Week Ending April 5, 2016

 * Task:

3/31/16 - Install gcc on majestix.

4/3/16 - Read logs.

4/4/16 - Learn more about sphinx config modifiers.

4/5/16 - Install tree on majestix.


 * Results:

3/31/16 - Successfully installed gcc on majestix. First I registered redhat. I then created the snapshot directory and the before and after snapshot directories orig and after-gcc. I then piped ls -al into a file for 15 directories. I then installed gcc. I then piped ls -al into a file for 15 directories after the install so they can be compared to the list of files before the install. I then did a diff on each file to see the differences. I found that only 4 directories changed. I created documentation of the installation process here: GCC Install Documentation

4/3/16 - Read logs.

4/4/16 - Looked into what modifications were done on previous experiments, also investigated modifiers online.

4/5/16 - Was unsuccessful in installing tree on majestix. This is because there is currently no internet access on majestix. I created before-tree and after-tree directories in the snapshot directory. I also piped 'ls -al' of the list of directories so they could be compared before and after the installation.


 * Plan:

3/31/16 - Install tree on majestix after emacs is installed.

4/3/16 - Continue reading logs.

4/4/16 - Continue investigating sphinx config modifiers. 4/5/16 - Installing tree on majestix once it gets internet access.


 * Concerns:

None

Week Ending April 12, 2016

 * Task:

4/9/16 - Create URC poster.

4/10/16 - Update URC poster after feedback from group. Read logs.

4/11/16 - Run train and decode of 0289/010.

4/12/16 - Read logs.


 * Results:

4/9/16 - Made URC poster for the group.

4/10/16 - Updated URC poster after feedback from the group. Read logs.

4/11/16 - Ran the train and decode of 0289/010.

4/12/16 - Read logs.


 * Plan:

4/9/16 - Get tree installed on majestix.

4/10/16 - Continue updating poster.

4/11/16 - Continue running trains and decodes.

4/12/16 - Continue reading logs.


 * Concerns:

None

Week Ending April 19, 2016

 * Task:

4/13/16 - Run experiment 0289/011.

4/14/16 - Run experiment 0289/014.

4/16/16 - Run experiment 0289/017. 4/17/16 - Run experiment 0289/018.

4/19/16 - Run experiment 0189/022.


 * Results:

4/13/16 - Ran experiment 0289/011.

4/14/16 - Ran experiment 0289/014.

4/16/16 - Ran experiment 0289/017.

4/17/16 - Ran experiment 0289/018.

4/19/16 - Ran experiment 0289/022.


 * Plan:

4/13/16 - Continue running experiments.

4/14/16 - Continue running experiments.

4/16/16 - Continue running experiments.

4/17/16 - Continue running experiments.

4/19/16 - Continue running experiments.


 * Concerns:

None

Week Ending April 26, 2016
4/23/16 - Research values to test in experiments.
 * Task:

4/24/16 - Read logs.

4/25/16 - Install tree on majestix.

4/26/16 - Read logs.


 * Results:

4/23/16 - Researched values to test in experiments.

4/24/16 - Read logs.

4/25/16 - Successfully installed tree on majestix. I first downloaded tree 1.7.0 from its source at http://mama.indstate.edu/users/ice/tree/ I then verified that tree 1.7.0 is i686 from here  https://www.archlinux.org/packages/extra/i686/tree/  I then uploaded the tree tar file to Caesar /tmp folder with winscp. During the process I was logged in as root on Caesar and as root on Majestix. I then copied the tar file from Caesar to majestix. I then recorded a snapshot of 15 directories before installing tree. I removed the tar file from the /tmp directory on Caesar. I then ran into a few errors when trying to install tree. The main errors were not using the correct options for the tar command with .tgz files, and also by not having the directory created before extracting. After extraction of the tar file I read the readme and install files. I then followed the install process and got tree installed. I then got a snapshot of the 15 directories after tree was installed. I ran a diff of the snapshot files and only three directories had any differences. I created a more detailed documentation of the installation process here: Tree Install Documentation

4/26/16 - Read logs.


 * Plan:

4/23/16 - Run experiments on findings.

4/24/16 - Continue reading logs.

4/25/16 - Running experiments.

4/26/16 - Continue reading logs.


 * Concerns:

None

Week Ending May 3, 2016

 * Task:

4/30/16 - Read logs. 5/1/16 - Read logs.

5/2/16 - Update my experiment information in the decode document.

5/3/16 - Research feature generation.


 * Results:

4/30/16 - Read logs.

5/1/16 - Read logs.

5/2/16 - Updated the decode document with my experiment information.

5/3/16 - Researched feature generation from paper.


 * Plan:

4/30/16 - Continue reading logs.

5/1/16 - Continue reading logs.

5/2/16 - Continue updating information.

5/3/16 - Continue research.


 * Concerns:

None

Week Ending May 10, 2016
5/4/16 - Install tree on Caesar.
 * Task:

5/8/16 - Work on Tools group report. Document tree usage instructions.

5/9/16 - Add more to Tools group report.

5/10/16 - Add more to report, update wiki.


 * Results:

5/4/16 - Tree is installed on Caesar without any dependencies and then updated documentation of tree on the install.

5/8/16 - Added information to the Tools group report. Added documentation of how to use the tree command here: Tree Usage Instruction

5/9/16 - Added more information to the Tools group report.

5/10/16 - Added more to report, updated wiki.


 * Plan:

5/4/16 - Continue updating documentation.

5/8/16 - Continue working on Tools group report. 5/9/16 - Continue working on Tools group report.

5/10/16 - Last class tomorrow.


 * Concerns:

None