Speech:Summer 2015 Trevor Downs


 * Home
 * Semesters
 * Summer 2015

Week Ending June 10th, 2015
The task this week was to assemble a detailed guideline  as to what I should be accomplishing during the  Summer 2015 semester (2CR), based off what Dr. Jonas discussed in the meeting. After submitting my version, Dr. Jonas revised/updated it.

The main tasks include:

 * 1.) New system configuration-Isolate: Brutus, Obelix and Idefix
 * 2.) Decode unseen test data from winning and losing team (from Spring 2015 semester)-- 256hr
 * 3.) Modify and/or create new prepareExperiment.pl to deal with difference for decode vs train
 * 4.) Document under "Run the Decode" to update new info
 * 5.) Update the two group pages (from spring 2015 semester - Bruins and Patriots)

Task #1 depicted below:




 * In order to prepare for the new system configuration task, I will be reviewing Spring 2015 Systems Group and also brushing up on Linux commands.

Task #1 - New System Configuration

 * Previous semester's documentation is pretty good, however it seems to be missing some details. I plan on re-documenting the process, and filling in details so it's more complete.

Task # 1 - New System Configuration
(6/18) Goal: Get Brutus up and running.
 * Directly connected to the console on the machine and booted it.
 * Had a disk error. Couldn't resolve the disk error.
 * Commented out that disk (that was giving the error). Removed it from the configuration file (fstab). Then upon that, the system came up.
 * Tried to look through /var/log .. to find out the issue with the disk. boot.log is empty. Analyzed other logs, no signs of any hints to why this issue is occurring.
 * Potentially have a broken disk. Will have to make a trip to the server room soon, and see what Brutus says upon start up. From there i'll be able to determine the issue hopefully.

(6/21) Goal: Fix disk issue
 * Attempted to try and find info pertaining to the disk issue via logs.
 * Found something that said "disk error" but there wasn't sufficient info telling what disk or anything specific.
 * used Fdisk and parted, to view disk hardware configuration. Found 1 disk.
 * used: vgdisplay, found no volume groups
 * used: pvdisplay, found no physical volume
 * used: lvdisplay, found no logical volume
 * Apparently, School is closed on Sunday's to students, so I wasn't able to get done what I wanted. Guessing this is just because it's summer. Waste of an hour driving.

(6/22) Goal: Fix disk issue & begin system configuration
 * Spent 2 hours in the server room.
 * On Brutus: resolved the disk issue (steps below)
 * Powered system down
 * Re-seeded drives (took them out, and put them back in)
 * Rebooted
 * Ctr A (to view adaptec). That verified that all disks were accessible.
 * Attempted to begin the new system configuration today, now that the disk issue is resolved.
 * Ran into some issues- Opensuse is a little goofy, iv'e never worked with it before. Will post results once I get more completed

(6/23) Goal: finish system configuration
 * Completed "task #1" New system configuration. Brutus, obelix and idefix are all up and configured as Dr. Jonas discussed (see diagram).
 * NSF mounted /mnt/main to Brutus. Changed the name to /mnt/main2. I was having an issue with this part. It was resolved by disabling the firewall on Brutus, because it was preventing the NFS mount. After the firewall was disabled, the clients were able to mount the servers disk.
 * Connected Obelix, Idefix, and Brutus to the switch. This was done through some trial and error. Since there was 2 cables attached to each machine (aside from Obelix because I knew that the redish/orange wire was to the internet and I was not to touch that). I had to try one, and if that didn’t work I tried the other and I got connectivity. Not sure what each cable did because they were both hooked up to the main switch.
 * Created new IP addresses for Obelix (192.168.11.3) and Idefix (192.168.11.3).
 * Edited the following files through VI editor. Everything was done through the xterm.
 * /etc/hosts (all machines)
 * /etc/fstab (all machines)
 * /etc/exports (Brutus only)
 * /etc/sysconfig/network-scripts/ifcfg-eth#
 * As I modified files, I made sure to create copies of the files as a precaution (using cp command_filename/year/month/date). Just in case I destroyed/broke anything. Luckily I didn’t break anything so those backups weren't needed.

New Task - Find out why Obelix isn't behaving (not listed in the original 3 I posted on the first week)
(6/24)New task for this week. After the meeting with Dr. Jonas, we discovered that obelix wasn't accessing the library. idefix has no problem accessing the library, and it runs the same operating system (RedHat), so this issue wasn't caused by the recent changes I made to the system. We suspect the issue is due to changes a past student made to the system. This week I will be doing some digging, investigating why obelix is having trouble accessing the library (lib /bin). Going to be reading through previous semester student logs, to find out exactly what changes were made. From there, the issue can be more easily diagnosed.


 * when the command "sphinx3_decode" is run on Obelix, nothing happens.

(6/25) Task: Begin investigating the issue.
 * Reading through Refik's log, since he did some work on Obelix last semester, that could be a potential lead on the issues that are occurring.
 * E-macs and gcc compiler was installed on Obelix last semester, wondering if this potentially has anything to do with the issue
 * Below is a picture of obelix's yum.log (what was installed recently)
 * Concerns: I cant ssh into Idefix. Not sure why, wondering if someone has been messing with the configuration. Because it all worked yesterday. (6/30) EDIT - Another student manually powered down the machine. I found this out by taking a trip to the server room. Although it was probably on accident, it definitely affected my progress

(6/26): Task: Continue investigation with Obelix's issue
 * Sift through /var/logs to see if there is any hints to the problem.
 * Ran some commands to search for problems.. these commands found a few messages.
 * demesg | grep -i error
 * demesg | grep -i abort
 * also ran ls -it | head (to find out what was recently modified).
 * Concerns: This issue is going to be difficult to resolve.

later in the day....
 * On obelix: tried running command "sphinx3_decode" .. command not found.
 * On Caesar: tried running the command "sphinx3_decode".. command was found.
 * After typing the command, found path of command - "type !$" ... "usr/local/bin/sphinx3_decode".
 * On obelix: went to the path of the command ("usr/local/bin/sphinx3_decode"), found link "local" (written in red).
 * next line typed: "ls -l".. showed "mnt/main2/local".
 * next: "grep main2 /etc/fstab" .. grep succeeded.. and found /mnt/main2.
 * tried to mount, "mount -a" and failed.. I suspect this is due to firewall preventing nfs mount.

Week Ending July 8th, 2015
(7/2): Solved the issue from last week, today. Obelix now communicates with lib/bin properly. Systems group from spring 2015, has good documentation on how to enable Sphinx. I used that as guidance
 * "/usr/local/lib" was missing from /etc/ld.so.conf, added it, then ran ldconfig command. This fixed the path issue, Sphinx is able to access libraries now.

Task #2 - "decode unseen test data"
Goal: Decode-the unseen test data from both winning team and losing team (from last semester- Spring 2015)
 * I will be creating directory "0280" everything I do will be in there.
 * In /mnt/main/corpus/switchboard :
 * Comparing full/train/trans/train.trans to 256hr/train/trans/train.trans
 * I will be running decode experiment using /mnt/main/corpus/switchboard/256hr/test dir

(7/3): Updated experiments page. Closed off Spring 2015, and created Summer 2015. Added my experiment directory to Summer 2015 (#0280). Now that everything is set-up, I plan on getting into the "bulk" of the work, in the next few days.

(7/6): In order for me to run the decode, I'm going to have to set-up my experiment directory (#0280) properly. This includes, run a train, and create a language model. So that is what I'll be doing prior to decoding. (i think, unless there's some way of just running the decode alone, but the directions on the wiki say that)

cd mnt/main/Exp/0280
 * Began running 256hr train. Steps taken below:

mkdir 001

cd 001

/mnt/main/scripts/user/prepareExperiment.pl switchboard 256hr/train

/mnt/main/scripts/user/generateFeats.pl

nohup scripts_pl/RunAll.pl. &


 * After these steps are completed, will update info---
 * Later in the day... Train seemed to hang- not sure why, all steps seemed to work. Waited a few hours, didn't get an error message or any kind of feedback that it was running, so I aborted it.

Created the language model from the directions via wiki, steps below:

Create LM direcory

mkdir LM

Enter directory for setup

cd LM

Copy transcript from corpus directory

cp -i /mnt/main/corpus/switchboard/256hr/test/trans/test.trans trans_unedited

Prepared the transcript

/mnt/main/corpus/switchboard/dist/transcripts/ICSI_Transcriptions/trans/icsi/ParseTranscript.perl trans_unedited trans_parsed

Copied the script that creates the language model.

cp -i /mnt/main/scripts/user/lm_create.pl.

Executed the script

./lm_create.pl trans_parsed

Week Ending July 15th, 2015
Goal: Assemble the proper files in my directory (#0280) in order to run a decode.

(7/9): Found out from yesterdays meeting that I don't necessarily have to run a train to run a decode. I'm going to copy files over (cp command) from previous experiments. This will be faster opposed to running a 256hr train, but the process might take me a little time to figure out what files are needed.

(7/13): Contacted a few members from Patriots and Celtics/Bruins (Losing and Winning team from spring 2015 semester), and I have had no luck gathering information from them. I need to find out what Experiment directory was used, so I can point my decode exp that way. Will reach out to Dr. Jonas and see if he possibly has any idea which directory(s) were used.

later in the day...
 * Contacted Dr. Jonas, he sent me the two teams reports. So now I know the directory(s)
 * I began the debug process on my decode.
 * First I looked at decode.log (cat decode.log)
 * At the top of the decode.log, there is a list with the paths of directory's needed to run the decode.
 * I checked each individual directory "ls " to make sure it was in my experiment.
 * A few things were missing, I tracked them down and added them into my experiment (#0280). From here I am getting two errors in the decode.log. I'll post them below:
 * ERROR: "cmd_ln.c", line 724: Cannot open configuration file /mnt/main/Exp/0280/001/model_parameters/001.cd_cont_1000/feat.params for reading
 * FATAL_ERROR: "mdef.c", line 680: No mdef-file

(7/14): E-mailed Dr. Jonas with questions about errors I was getting Concerns: Attempted to copy feats directory from previous semesters class however my session timed out, putty booted me. This happened twice so not sure where to go from here. Will ask Dr. Jonas during the meeting today (7/15)
 * Fixed: ERROR: "cmd_ln.c", line 724: Cannot open configuration file /mnt/main/Exp/0280/001/model_parameters/001.cd_cont_1000/feat.params for reading
 * modified script: run_decode.pl
 * the path was incorrect, 001.cd_cont_1000 (was missing a "u" - "count"), changed it to 001.cd_count_1000. Now that error is resolved. Onto the next one.
 * EDIT: as of 7/16, Dr. Jonas made a correction, the path is supposed to be "cont", so it was right originally
 * New error:
 * SYSTEM_ERROR: "corpus.c", line 562: fopen(/mnt/main/Exp/0280/001/etc/001_decode.fileids,r) failed ; No such file or directory
 * Fixed error: went into mnt/main/Exp/0280/001/etc.. renamed 001.decode.fileids to 001_decode.fileids (mv command)
 * New Error(s):
 * SYSTEM_ERROR: "pio.c", line 450: stat(/mnt/main/Exp/0280/001/feat/sw2289A-ms98-a-0002.mfc) failed; retrying... ; No such file or directory
 * ERROR: "feat.c", line 387: stat_retry/fopen(/mnt/main/Exp/0280/001/feat/sw2289A-ms98-a-0002.mfc) failed
 * FATAL_ERROR: "utt.c", line 120: Cannot read file sw2289A-ms98-a-0002. Forced exit
 * Assuming this is related to feats directory

Week Ending July 22nd, 2015
New plan for this week, instead of continuing with attempting to copy all files from previous training experiments, Dr. Jonas wrote a new script ./run_decode_v2.pl. I am going to try to get this to run. What this script does is, it takes a decode directory (first argument) and a train directory (second argument). Meaning in order to run the decode script, it wont have to have the train directory within the same experiment directory as the decode. So i'll essentially be able to run a decode on a past experiment without copying all the old files into my decode directory.

(7/16): Today, I got the decode on unseen data to run successfully! (with the help from Dr. Jonas). Now time to score.. [root@caesar etc]# sclite -r 001_train.trans -h hyp.trans -i swb >> scoring.log Error: Not enough Reference files loaded Missing:
 * First attempt at scoring the decode
 * Solution: convert transcript from corpus to reference. (This should solve the problem)
 * At the very bottom of this page, there is a troubleshooting guide to solving this issue

Contents of scoring.log below:

SYSTEM SUMMARY PERCENTAGES by SPEAKER

,-.     |                            hyp.trans                            | |-|     | SPKR    | # Snt # Wrd | Corr    Sub    Del    Ins    Err  S.Err | |-+-+-|     | sw2289a |   50   1220 | 11.6   17.5   71.0    3.8   92.2  100.0 | |-+-+-|     | sw2289b |   48    962 | 12.6   21.8   65.6    5.5   92.9  100.0 | |-+-+-|     | sw4361a |   51    756 | 16.5   26.7   56.7   10.7   94.2  100.0 | |-+-+-|     | sw4361b |   27    324 | 19.1   25.6   55.2   21.9  102.8  100.0 | |-+-+-|     | sw4379a |   41    432 | 20.4   20.8   58.8   25.2  104.9  100.0 | |-+-+-|     | sw4379b |   39    828 | 10.4   19.2   70.4    3.4   93.0  100.0 | |=================================================================|     | Sum/Avg |  256   4522 | 13.8   21.2   65.1    8.6   94.8  100.0 | |=================================================================|     |  Mean   | 42.7  753.7 | 15.1   21.9   63.0   11.8   96.7  100.0 | | S.D.   |  9.1  333.0 |  4.2    3.6    7.0    9.6    5.6    0.0 | | Median | 44.5  792.0 | 14.6   21.3   62.2    8.1   93.6  100.0 | `-'

Task #3: Modify and/or create new prepareExperiment.pl
(7/17): Goal: This week, create/modify a new prepareExperiment.pl
 * Purpose is to deal with difference for decode vs train. Since prepareExperiment.pl is currently set-up for training. The modifications will be made so that there is another separate prepareExperiment.pl (script names tbd) for decode as well.
 * Script path: /mnt/main/scripts/user/prepareExperiment.pl

(7/19): Today I spent time analyzing the script (/prepareExperiment.pl), in order to gain a better understanding of how it operates. Now that I have a more solid grasp, I plan on creating a new script later in the week. I'll be offline the next few days but once i'm back, i'll be updating my progress.

(7/21): Set-up scripts/prepared
 * Renamed the old /mnt/main/scripts/user/prepareExperiment.pl..
 * created a copy using cp command. Copy of original "prepareExperiment.pl" = "7212015prepareExperiment.pl" (dateFilename).
 * The train experiment was renamed to: "prepareTrainExperiment.pl"
 * The decode experiment will be called: "prepareDecode.pl"

Week Ending July 29th, 2015
(7/26): The past few days I have been assembling team e-mails from the 2015 spring semester (Patriots and Bruins). Since I haven't had internet access, only the e-mail documents locally. This is what I've been able to work on.
 * Organizing e-mails in a Word document, to make them more readable. (deleting all unnecessary info, only keeping content, name and date).
 * Will be adding the info to the team page wiki's once the document is completed. To be continued...

(7/28): Edited How to run a Train page
 * Modified create directory structure: script name was changed, new name below:
 * /mnt/main/scripts/user/prepareTrainExperiment.pl switchboard first_5hr/train

(7/29): Worked on Bruins page (winning team from spring 2015 semester), added their content and began formatting it to wiki standards. Still needs a little more formatting which will be done in the upcoming days. Since I was on the other team, it's interesting reading their e-mails- they did a lot of spying on our team- so i don't feel as bad about losing (Since we didn't try to "cheat").

Week Ending August 5th, 2015
(7/30): Today I completed the two Team Spring Semester 2015 pages, Bruins,  Patriots.
 * Formatted team documentation (word documents) to Wiki standards, in order for it to look easier on the eyes. This took some time, mostly just repetitive work that a monkey could do.

Future suggestions: Next semesters teams should submit their team documentation (e-mails and such) to a wiki page. Just like how the proposal and final report were done. Just in-case there isn't a student taking the class immediately after, like me, whom has to re-document everything to the wiki. That way team's can't leave the class without proper documentation, they will be forced to upload it.

(8/2): Finished updating "Run the Decode" section.
 * Created 2 pages under the section, to reflect the differences between decoding on trained vs unseen data
 * Run a Decode on Trained data: click here
 * Run a Decode on Unseen data: click here