Speech:Spring 2014 Justin Silva Log


 * Home
 * Semesters
 * Spring 2014
 * Proposal
 * Report
 * Information - General Project Information
 * Experiments - List of speech experiments

Week Ending February 4th, 2014
1/31/2014

Read other logs and brushed up a little on the specs of the tools.

2/2/2014

Read other students logs who were not in my group so I could get a more general knowledge on the project as a whole.


 * Task:

2/3/2014

To learn about the current tools in use by the servers.

Log in and change my email on the server I have been assigned.

2/4/2014

Still haven't figured out how to log into the Caesar server, I was hopefully planning on trying after class on Wednesday the 5th but it seems we might have a snow day. I will ask my peers to get help learning Linux and logging in.


 * Results:

2/3/2014 I have learned that the current operating system on the Caesar Server and the 9 other drones is Open SuSE. Open SuSE is an open source operating system that includes various ideas from other versions of linux. The version we have installed on the servers is 11.3 Open SuSE 11.3 is no longer being supported by its developers and the newest version is 13.1 (So we are 2 versions behind) I feel tho because the software runs fine on the current version of the OS it should not be an issue and there is no reason to update.

2/3/2014 For the speech part of the server we use an Open Source package called Sphinx. We currently have version 3 installed on the server. Sphinx is a tool kit and contains multiple parts like a trainer and a decoder. There is a newer version on sphinx, but version 4 is a completely new program so existing experiments / tests / scripts could possibly be nulled if we decided to upgrade to 4.


 * Plan:

2/3/2014
 * Concerns:

It would be interesting to see how Open SuSE 13.1 and Sphinx version 4 would work together and if we could transfer all old content to the new versions (this would be a huge task and I'm a firm believer of "if it is not broke, don't fix it"

Week Ending February 11, 2014

 * Task:


 * Results:


 * Plan:


 * Concerns:

Week Ending February 18, 2014
Look into backing up the tools and information
 * Task:

2/15 Checked over the logs of my group partners to get caught up with the current status of the group.
 * Results:

2/16 Read some more logs

2/17 I learned that there is a version 2.05 of the CMU Toolkit. We are currently using .7 in the lab so we could upgrade to version 2.05

The documentation for Version 2.05 can be found here

this is an excerpt from the Version 2 Documentation Version 2 of the toolkit seeks to maintain the structure of version 1, to include all (or very nearly all) of the functionality of version 1, and to provide useful improvements in terms of functionality and efficiency.

2/18 The Carnegie Mellon University Pronouncing Dictionary is a machine-readable pronunciation dictionary for North American English that contains over 125,000 words and their transcriptions. This format is particularly useful for speech recognition and synthesis, as it has mappings from words to their pronunciations in the given phoneme set. The current phoneme set contains 39 phonemes, for which the vowels may carry lexical stress.

Note that this dictionary is not in the format you want to use for the Sphinx decoder. For that purpose, either go to SourceForge or use the following tool. 

Online LM Tool, produces a dictionary which matches its language model. It uses the latest CMU dictionary as a base, and is programmed to guess at pronunciations of words not in the existing dictionary. You can look at the log file to find which words were guesses, and make your own corrections, if necessary. With the advanced option, LM Tool can use a hand-made dictionary that you specify for your specialized vocabulary, or for your own pronunciations as corrections. The hand dictionary must be in the same format as the main dictionary

(2/17) I will look into how to perform a back up of the current tools we use in the class. Also, will look at CMU Dictionary .7a combined with LMTool Converter
 * Plan:
 * Concerns:

Week Ending February 25, 2014

 * Task:

(2/22) Logged in to Caesar server from home using caesar.unh.edu using the PuTTY program. Also downloaded VMWare Player and am currently installing openSUSE 13.1 because i feel it will be better to try to implement tests on my personal rig then messing with the actual Rome server.

(2/23) Logged in to read some logs and see where the other groups were at. Still trying to figure out the way to copy to local folder but not have it save in the caesar main directory. For some reason when I'm trying to run my Virtual Machine i keep getting an error saying "Failed to power on: Internal Error" (Justin Alix sent me a different link for openSUSE so i will try installing that and testing it)


 * Results:

(2/24) From the ReadMe Documentation from CMU SpinxTrain 1.0.8 (Newest Version not currently installed)

SphinxTrain 1.0.8

^^^^^^^^^^^^^^^^^

New features:

* New openfst-based G2P trainer and decoder, supported by Sphinx4 too. * Parallel feature extraction * Package can be installed now just like any application * Single 'sphinxtrain' command to access all training process * Increased reuse of sphinxbase functions Fixes: * Memory leaks and build issues

The documentation reads that it can be direct upgrade and if we decide to go to Sphinx4 it will be compatible with it. I think it could warrant an upgrade but really only if we decide to go to Sphinx4

(2/25) For SCLite v2.9 you can only download it in a package from http://www.nist.gov/itl/iad/mig/tools.cfm The current toolkit that it is included in is the Speech Recognition Scoring Toolkit v2.4.8 I read over the read me for the Toolkit and it has not been tested on OpenSUSE 13.1 it has been tested and compiled on a few other linux systems but it did have errors depending on the version and maker.

If we decided to upgrade it would take extensive testing before going live in my opinion.


 * Plan:


 * Concerns:

Week Ending March 4, 2014

 * Task:

(2/28) Read some logs and still trying to figure out why VMWare will not work on my personal computer. hopefully it gets resolved soon.
 * Results:

(3/2) Still cant use VM ware on my system, looks like im going to have to try using my groups server to test new software on, will be difficult because im not sure how to install an OS through the internet or even command line, will look into it, might have to do a true install off a CD and at the actually server. The Problem with my system keeps stating that my processor is not compatible with 64 bit OS's on VM Ware even though my windows 7 is a 64 bit version.

(3/3) Installed clean version of openSUSE 13.1 and clean version of VMPlayer and still having issues installing it. looks like it will have to all be done on our groups server which is very unfortunate.


 * Plan:


 * Concerns:

Week Ending March 18, 2014

 * Task:

(3/8) Logged in to do some logs, also did a little research on how to break the link from our computer to Caesar. will have to followup with Prof. Jonas because im very unsure about how to use the rm UNIX command and I don't want to accidentally delete everything and ruin all peoples work (3/17) Trying to run a basic train currently using exp folder 0213 for my testing. Currently hung up on finding a current script will have it updated soon!

(3/18) started to work back on my first time running a train, for some reason it hangs up like it is trying to find a script but the script is in the wrong location. still trying to work it out. will have an update soon!


 * Results:


 * Plan:


 * Concerns:

Week Ending March 25, 2014
(3/22) Logged in to review my new teammates logs
 * Task:

(3/23) Today i tried running a train with the new master script. my experiment number was 0232. I used a senome value of 5000 and a density of 128.
 * Results:

I ran the train successfully and the creation of the Language Model. But for some reason when i go to decode nothing happens. i run the Decode using this command   ./run_decode.pl 0232 0232  This is also run under the location at   ...0232/DECODE  and it doesn't return anything. Will work on this experiment again to figure out why its not working right now.

(3/25) Did some research on Sinisa's problem with a **write failed: broken pipe**. I believe it is a problem with the SSH config file having an early disconnect if idle for to long. there were alot of people on some forums with this issue and some of the resolutions people gave were to edit the SSH file and to add a line or change the line to look like this ** ServerAliveInterval 30 ** another option would be to run the program in the background and then just go back a few hours later, relogin and check the decode file.


 * Plan:


 * Concerns:

Week Ending April 1, 2014

 * Task:

(3/27) Broke the link from trebadix from Caesar so we can start installing new software and testing. This will be good when we start working on it because we can do anything we want and it will not directly affect Caesar. the downside is everytime we SSH into trebadix now we have to log in as root, but its not a big deal.
 * Results:


 * Plan:

(3/30) Logged in to check classmates logs

(3/31) Did alot of research on how to install the new software when sphinx 4 is up and running on the trebadix node

(4/1) Worked on trebadix a bit, trying to figure out where everything will be installed, and learning the server as it is while it is unmounted from the regular caesar server, also was messing around with how to mount and unmount from the caesar server using the FSTAB file. The FSTAB file is located in the etc folder under the main directory. to mount and unmount it is just a simple vi fstab, then move down to the caesar:/mnt/main line and either comment it out or delete the comment.
 * Concerns:

Week Ending April 8, 2014
(4/4) Logged in to view some logs, also look at what the new group is doing. trying to figure out how to be able to view my groups Google page because the email that was activated was not a Google account. Sent Pauline a message to re invite me on a Google account.
 * Task:

(4/6) logged im to report that I couldnt make it to the group meeting because I was only notified about it on saturday and I was working all weekend. Still cant get in touch with Pauline about joining the google group. Ive tried sending 2 emails and a facebook message. Will keep trying tho. I did look over the brief summery that ray sent the group tho about what the meeting involved.


 * Results:


 * Plan:


 * Concerns:

Week Ending April 15, 2014
(4/12) Logged it to report that i have been working on the Tools Groups poster for the URC Event. Will have it finished hopefully tomorrow. Also viewed some of my other groups logs from Justice League.
 * Task:

(4/13) Finished up the tools poster, waiting on Justin Alix reply on what I can change on it to make it better or easier to present. Been reading what my group is up to in our google groups log, this week is mainly research for us to figure out what the best parameters are that we can use. Will be doing more research over the next few days to have a good report for my team.

(4/14) Did some research about how weighting affect outcome of the word error rate that Forest brought up in our group discussion last wednesday. it appears that weighting can actually help because it goes to the more relevant choice if it cant get a word perfect. the one thing i did notice in alot of posts to forums on the internet tho is that sphinx doesn't support weighting. will have to go more into it tomorrow and report to my group about my findings. if we can get weighting to work with sphinx that may dramatically lower our word error rate, but it might be unplausable in a 100 hour transcript for how short of a time we have.
 * Results:


 * Plan:


 * Concerns:

Week Ending April 22, 2014
(4/19) took a look at my new groups logs, also looked at the new scripts that david wrote for us to fix some of the errors we were having on our train
 * Task:

(4/22)Looked at my groups logs to see where there at.


 * Results:

(4/20) Ran a train under the folder 009 using mini/train. Was using davids new script that uses links to the audio files instead of actually placing the files in the folder, also set the senone value to 500 because its only an hour of audio, also set the density for 1 to 64. but i ran into an issue trying to run the other script "nohup scripts_pl/RunAll.pl . &"  will have to touch base with David on where i should be running this script from. sending him an email now!

(4/21) Worked with David to figure out where it was messing up, apparently there was an audio file that was not being read correctly when it was being read. this is figured out now tho, will be trying to run this again tomorrow when i have a free minute. the rest of Davids scripts work perfectly.

(4/22) Ran another train using the updated scripts under the folder 013, This tran used the first_5hr/mono transcripts.


 * Plan:


 * Concerns:

week Ending April 29, 2014

 * Task:

(4/26)Logged in to view justice leagues logs.

(4/27) did some looking into why some peoples test experiments were not working.


 * Results:

(4/28) Ran a train on first_5hr using Senone 2000 and a density of 64. Will post the results as soon as soon as the decode is done.

(4/29) I had to do my decode twice so it took a while because last night a fuse popped and the server went down

The results i got were

TOTAL Words: 60084 Correct: 54688 Errors: 11149 TOTAL Percent correct = 91.02% Error = 18.56% Accuracy = 81.44% TOTAL Insertions: 5753 Deletions: 2433 Substitutions: 2963


 * Plan:


 * Concerns:

Week Ending May 6, 2014

 * Task:


 * Results:


 * Plan:


 * Concerns: