Speech:Spring 2013 Thomas McCarthy Log


 * Home
 * Semesters
 * Spring 2013
 * Proposal
 * Report

Week Ending February 5th, 2013
-Feb 3: Review Cedric's notes on experiments and running training.
 * Task:

-Feb 4: Run train

-Feb 3: Reviewed the notes, attempted to mkdir on Asterix and Obelix...but could not :( I read other notes and logs.
 * Results:

-Feb 4: After talking with Eric, we determined we cannot write to any of the servers but Caesar. I began to follow the steps to figure out how the training is run and executed. Posted in Group

-Feb 5: I read logs this evening to further understand the project


 * Plan:

-Feb 4: Some of the required files (trans_unedited) do not exist. I need to look into finding these to attempt the train!
 * Concerns:

Week Ending February 12, 2013
-Feb 7: We hung out on Google+ to run a tiny train together -Feb 10: To read student logs -Feb 11: To run my own tiny train
 * Task:

-Feb 7: After some tweaking our dictionary, Eric was able to run a tiny train -Feb 10: I read logs -Feb 11: Work ran late, school classes took over, I spent some time this evening reviewing logs and the documentation.
 * Results:

-Feb 7: We will run a tiny train to test our ability/proficiency. -Feb 10: I will attempt to run my own tiny train tomorrow -Feb 11: Run the tiny train tomorrow and see what I need to modify to get it to run and decode (if possible)
 * Plan:


 * Concerns:

Week Ending February 19, 2013
Feb 16: Read logs
 * Task:

Feb 17: To get a train almost ready to be run to understand the process and scripts better.

Feb 18: See results, because there was no task within Caesar I can currently do directly related to my previous plan!

Feb 19: To resume testing

Feb 16: I read logs, especially Eric's, since he seems to be the most educated on the project!
 * Results:

Feb 17: I tried running 0021, but I'm apparently an idiot. The line "/mnt/main/scripts/train/scripts_pl/pruneDictionary.pl 0015_train.trans /mnt/main/corpus/dist/cmudict.0.6d 0015.dic" gives me trouble when I change it to 0021. I get "sh: text2wfreq: command not found" and "cat: 0021_train.trans: no such file or directory". I reached out to my group peers on Google+ and via email to get some insight for tomorrow. I attempted to try to work around the issue, but I cannot figure out what I missed that caused it to happen.

February 18: I spoke with the Group on G+ and Eric let me know what there is an issue with the Sphinx installation, or more specifically the decoder, which will not compile due to an undefined reference. This would explain the error I'm receiving with the pruneDictionary Perl script that failed yesterday. Eric also encountered the same issue. I also checked with Tyler to see if he had any input/idea, and he said he spoke with Eric this evening as well and was notified of the Sphinx installation error. I also familiarized myself with the CMU Sphinx Sourceforge website, including reviewing their documentation to see if it would benefit my understanding and familiarity with the whole modeling process. Unfortunately, they did not have much (they blame it on lack of end-user tools), but I did see some of their examples that they explain and demonstrate. Also, here's an interesting (although basic) article about Google's awesome speech recognition in Android 4.1.x/4.2.x Jellybean.

Feb 19: Caesar is seemingly unusable to anyone. I cannot connect to it and get an error message. Eric posted about this this evening on G+. I checked after work, around 10pm, with no avail. Instead, I read logs!

Feb 16: I will work on continuing our train tomorrow evening.
 * Plan:

Feb 17: I am going to contact Eric to see where he has come along with the decoding to attempt to assist him before class meets Wednesday. In addition, as I stated, I am looking to get feedback from my peers on the issue I ran in to and seeing where Eric is.

Feb 18: From our conversation on G+, Eric said he was going to contact Mike for some advice. I also suggested we meet up on a G+ Hangout tomorrow to try and tackle the compiler errors we're receiving together. Hopefully with all/some of us working together, we may be able to recompile the whole installation and/or fix/compile the Sphinx Decoder to make it function correctly again. Feb 19: Since we meet tomorrow, we'll probably discuss the issues with Caesar and catch up all together as a group and a class.

Feb 17: As I stated above, I'm concerned about not being able to run a simple tiny train on my own yet. I hope that by tomorrow I'll be able to, and then discuss goings-on with the rest of the group to begin chugging away at the project moreso!
 * Concerns:

Feb 18: Obviously Caesar unable to run models/trains/etc in this state is a problem we need to solve [hopefully] tomorrow, or possibly in class on Wednesday to make sure that we can further our understanding and proceed learning more about the modeling process, including eventually expanding into larger trains.

Week Ending February 26, 2013
-Feb 25: To catch up on logs and some stuff on G+
 * Task:

-Feb 26: To run a train (yes, it'll technically be the 27th, but look at the initial time i wrote this!)

-Feb 25: I read up on the logs within the Wiki, and I also read Eric's post about his Perl script he created to be able to run and decode a train much easier!
 * Results:

-Feb 26: I successfully ran a train. This is good, because we've always run into issues (read only, Caesar broken, etc). Now we will work on simplifying the process for other groups to do, which Eric has began with his new script he made which he posted on Google+. This way we can cut down much time when running a train.

-Feb 25: Tomorrow at some point I plan on utilizing and reviewing his script to help out the train/decode process.
 * Plan:

-Feb 26: Tomorrow at the meeting, I will discuss Eric's script he wrote and we will determine how/whether/when to incorporate it into the directions within the wiki. It is an immensely helpful script that will make a great difference. -Feb 25: My only concern is more of the fact that I'm unable to make 4 posts this week. I've been too busy. So tonight I had time, and tomorrow I will as well. :/
 * Concerns:

Week Ending March 5, 2013
-March 2: To read logs and catch up with other modeling group members -March 3: To work on scoring
 * Task:

-March 2: I read some logs and also posted to G+ to see where the other members are at in terms of progress, especially since we didn't have a meeting Wednesday due to the horrible weather. -March 3: Following suit to Eric, I attempted to score 0020 vs. 0015 to see what I could decipher from the errors we received. They also kept appearing over the missing transcripts as they did with Eric. I will be in contact with everyone to see if/how anyone knows to remedy the errors that we encounter. From there, hopefully we can chug along with succeeding in scoring the experiments. This will get us to a solid point within the modeling process. -March 5: This evening I followed up to responses on G+ and read logs
 * Results:

-March 2: Tonight/tomorrow I will watch G+ and respond to figure out what we should all focus on next, whether it be still training, or furthering Eric's script he wrote and continuing to streamline the whole process! -March 3: Tomorrow I will either read logs or do some more decoding/scoring work. Once we tackle the scoring errors, we will be at a good point to evaluate what work we're doing and how best to optimize it, especially for other teams. Eric did a good chunk with his new Perl script, but surely there's more to be done in that respect. I also sent out a mass email to coordinate a meeting with Mike hopefully before Wednesday, since we're not all responding timely on G+! -March 5: We will be having our much-needed meeting tomorrow where we'll discuss the current state of affairs and what's next. We also need to Skype with Mike sometime.
 * Plan:
 * Concerns:

Week Ending March 12, 2013
-March 7: To read logs after the meeting
 * Task:

-March 7: I read logs, specifically looking at how Caesar is doing.
 * Results:

-March 7: I will continue to work on training and decoding.
 * Plan:


 * Concerns:

Week Ending March 26, 2013
-March 23: To read logs -March 24: To hang out and run a train with the new group. -March 25: To read logs -March 26: To run a train again on Obelix to prepare for decoding and scoring in class tomorrow with the group
 * Task:

-March 23: I read logs -March 24: We hung out for about an hour and a half and attempted to run a train. We ran into a few issues with the new directions that we have to use. Scott had an issue with his path /mnt/main/local/bin/ Next thing: genPhones.csh gets copied over to the directory, then called the .pl file. We changed this to the .csh script. In addition, the script does take a parameter--the experiment number. I modified the Wiki to reflect that. Once we got to making Feats, I encountered this error while running make_feats.pl: "Failed to open param file ./etc/feat.params for writing: Permission denied at /mnt/main/scripts/train/scripts_pl/make_feats.pl line 128.". Upon further inspection, it was trying to write to my root directory on Obelix. Scott did not have the same issue on Miraculix, nor did Tyler on Methusalix. We ran into some words missing from the dictionary, which were easily remedied. Eventually, I was not able to get runAll.pl running on Obelix, nor could Scott (Miraculix). Tyler's runAll.pl did run on Experiment 0031 on Methusalix. -March 25: I read logs and Eric's Google+ post he shared with me regarding the errors and how he believes he remedied them. -March 26: I was able to follow the steps on Obelix and my edits and Eric's changes solved the problems we encountered during our hangout the other day.
 * Results:

-March 23: Tomorrow my new group and I will be Hanging out to learn! -March 24: We'll need to make sure we can get these issues minimized so that training will be less problem-prone. It's almost like a game of cat and mouse at present. I'm going to contact Eric today or tomorrow and see what he thinks caused the weird issues we had. -March 26: Tomorrow in class, our group will continue our Hangout and decode and score the train. We will probably completely redo the modeling as well so that it will work as a "end to end" flow. -March 24: It's clear that some of the servers are configured differently from one another, since we were having different errors that arose at different times with different scripts. This could be an issue as the class progresses and more trains (and decoding/scoring too) are processed.
 * Plan:
 * Concerns:

Week Ending April 2, 2013
-March 27: Meet in class and discuss the group's progression.
 * Task:

-March 28: To read logs

-March 31: To hangout again

-March 27: We met up in class in our group and talked about our previous Hangout and our problems we encountered. In the class meeting, Eric discussed what errors and what the possible problem could be. We decided to meet up next Sunday night (March 31) to continue into the train and decode.
 * Results:

-March 28: I read logs to see how the other groups were doing with their modeling attempts. I also determined that I would run a mini train before Sunday to make sure things can go smoothly!

-March 31: We hung out and discusses progress. Scott had started a decode before we met up, and was able to score it.

-March 27: The plan is kinda the results!
 * Plan:

-March 28: I plan on running a mini train before Sunday to make sure everything will run smoothly on a new experiment!

-March 31: I am going to run a mini train, decode, and eval on Obelix before class Wednesday and document errors and what I did as much as possible to make sure by Wednesday everyone can be on the same page

-March 27: The only concern would be encountering any issues with our next modeling attempt, though according to Eric we should be all set!
 * Concerns:

Week Ending April 9, 2013
-April 3: To meet in class and catch up with what's going on with members.
 * Task:

-April 7: To Hang Out or read logs, depending on group feedback

-April 8: To make sure the group is on the same page and everything's OK.

-April 9: To follow up with a few of the trains being run, including the 5 hour one.

-April 3: We were given the task of working on a 5 hour train. Within class, I helped Bego get more familiar with running a train. We made great progress and as far as I know he should be almost all set. We got to decoding.
 * Results:

-April 7: Earlier in the day I sent out an email to the group (well, to Bego & Matt, and cc'd other members) to see if we needed to Hang Out. We didn't Hang Out, and Bego has been in contact with me about the Decode we began in class the other day. So, this evening I read logs.

-April 8: I emailed Matt regarding the project we're supposed to be working on with the other group (splitting a train). He was splitting up the data so that we could run it piece by piece. In addition, I emailed Drew to see how his group was; to see whether they had begun the train on their own or not. In regards to catching up other stragglers, Bego Google Talked me last night and today about issues he was running in to while decoding. He kept getting disconnected. He tried it while at UNHM (not sure of the status of that run yet)

-April 9: Drew responded that he was waiting on Matt for the 5 hour train. Later today, Tyler told me via SMS that it was ready and he began it a few hours ago. There were 111 words missing from his dictionary and he added them in (Exp 0078). Meanwhile, Bego contacted me about Experiment 0070. He was able to complete the Decode the other day without a hitch (we think his computer was closing the connection). However, during scoring, he was getting "hypothesis file hyp.trans does not exist." Looking into it, the parseDecode.pl script executes a shell command to make a temp.log file and it fails writing it. I tried making it manually via vi and nano and it didn't work. I eventually logged in as root on caesar and the script was able to create the file. It ran, but then I ran into missing reference files galore!

-April 3: To make sure all members of the group are all caught up.
 * Plan:

-April 7: Tomorrow I'm going to email everyone to make sure we're all on the same page and ready for class Wednesday!

-April 8: Maintain contact to make sure everything's falling in to place with the group. At this point it seems everyone is caught up or working on something important to furthering the class (Matt)

-April 9: Tomorrow in class we'll all probably run the 5 hour train together. Or at least demonstrate what it takes to do it!
 * Concerns:

Week Ending April 16, 2013
-April 13: To run a train via Hangout if necessary and to catch the group up. -April 14: To read logs -April 15: To hang out and run a community train
 * Task:

-April 13: Charlie began a large email conversation with groups AD to see what we needed to do. Some members gave some input and we ended up not being able to coordinate anything, instead opening the line for issues that arise while members are running the train.
 * Results:

-April 14: I read logs to reach out to people who are struggling. I also sent out an email to the group to see everyone's status so far.

-April 15: We hung out beginning at approximately 9:15pm. Many members came: Tyler, Matt, Bego, Mike, Justin, Drew (Nick watched on YT. Sam, Vinnie, Jake, Charlie did not join the Hangout, but may have watched Live on YT). While running the train (which we didn't get too far...as I write this it's been 3 hours. We ran into a few issues with the new genTrans script especially in regards to forward slashes in the regular expressions. The next issue we ran into was in regards to hyphens. The current script removes the hyphens, leaving one massive word, which messes up the dictionary. We ended up with 27 words that needed to be added into the dictionary, which Tyler took care of. However, as I was typing this (feels like a narration now), Tyler went to go run make feats, and got a script not found error. He noticed that /mnt/main/scripts/scripts_pl was missing. The folder is deleted. We looked around and Tyler found (what seemed like) a backup From 5am on April 10. This is better but still not good that the folder was seemingly deleted... AAAND we're back, when we were copying the "backup" over, it turns out the scripts_pl was in the user folder under scripts. So that's good. *phew*

-April 14: I will be in contact with the group to make sure we get our goal done and that nobody is left behind in running their own train or the group's train. -April 15: I really want to make a script that can add in missing words with pronunciations automatically. Not even sure if this is reasonable yet. I may get to that. Otherwise, I'm going to email the other group members to give everyone the YouTube
 * Plan:
 * Concerns:

Week Ending April 23, 2013
-April 17: In class work on poster and coordinating code for the group for the week -April 21: To read logs -April 22: TO read logs
 * Task:

-April 17: We worked together on the poster and attempted to coordinate the coming week for modeling -April 21: I read logs and followed the email chain in regards to the dictionary we're using. -April 22: Tonight I read logs because Caesar is still down.
 * Results:

-April 17: We will continue to stay in touch via email as group AD to coordinate modeling and dictionary creation. -April 22: Depending upon Caesar's health, I may not be able to do much more than read logs tomorrow! We'll see
 * Plan:


 * Concerns:

Week Ending April 30, 2013

 * Task:


 * Results:


 * Plan:


 * Concerns:

Week Ending May 7, 2013

 * Task:


 * Results:


 * Plan:


 * Concerns: