Speech:Spring 2019 Aashirya Kaushik Log

From Openitware
Jump to: navigation, search


Week Ending February 4th, 2019

Task

The first task for me for this week was to be able to login into to Caesar using the UNH VPN. I was initially having issues with the Putty on my windows machine but, now I have that resolved and I also decided to not use the windows machine and instead use mac for this project because it is so much easier to navigate. My next to do thing was to read up on the project as much as I can to understand the requirements in detail for me to be able to run my first experiment. I also plan to read up more about the experiments done by the data group in the past couple of years. I also plan to read up more about the Sphinx and get more comfortable with it. I also plan to run my first experiment. Get more comfortable with Caesar and navigate using the UNIX commands. I also planned to read other people's logs.

Results

Successful login to the Caesar using VPN. I did read up on Sphinx which helped me understand how the speech recognition actually works and what role Sphinx plays in it. I still havent been able to run my first experiment successfully as I spent most of time this week doing the readings about what was done by the data group last year and also trying to understand Prof Jonas's expectations from our data group this year. I did create a sub directory 006 under the 0313 for my first experiment. I was able to add an experiment number 015 using the addExp.pl -s as Peter suggested in our group meeting how using -s is better than using -r. I am still in the process of running my experiment, I should be able to do it by tomorrow's class. I also tried to ssh to the astrix but, due to some reason I was not able to. I also plan to investigate on that this week.

Plan

Read up more about the experiments and about the project, meet with my group to come up with a plan to fulfill the expectations from the data group this year.

Concerns

At this moment, I am mostly concerned about being able to get hold of the project and be able to achieve what is expected of me and my group.

Week Ending February 11, 2019

Task

02/05/2019 - Finished running my first experiment that was created in the sand box that Prof Jonas created for the class and posted the score table on the wiki for everyone to see. Started reading and looking into last year’s experiments 0305/011,012,013

06/05/2019 – Met with the group in school and discussed who would do what and we made sure that everyone was on the same page. Peter volunteered to work on the proposal and he mostly did the work on the proposal, we all reviewed the paper on Friday and finalized it. Monica was having some issues with her decode for her first experiment. She was getting the following error:

"Not enough reference files loaded, Missing:"

We learnt from the wiki notes from last years group as they stated "that this error is caused by duplicate identical transcript entries in either the hypothesis transcript and/or the reference transcript. Usually it is the hypothesis transcript that causes the error" . We as a team tried to resolve the error by following the steps that were mentioned in last years notes. These steps from the last years notes are as follows:

  1. Go to your experiment's etc directory if you aren't already there.
  2. Remove all redundant lines.
    • We use a built-in Unix tool called uniq to do this for us. The output of this tool needs to go to a new file.
    • % uniq hyp.trans >> hyp.trans.uniq
  3. Restart SCLite while using the newly created hyp.trans.uniq file.
    • sclite -r <experiment #>_train.trans -h hyp.trans.uniq -i swb >> scoring.log

It still didn’t work and she decided to meet with Prof Jonas to figure out what the error was.

07/05/2019 – One of the tasks assigned to the data group was to rerun the 5 hour train and then decode on the following experiments from last year 0305/011,012,013. After having my first experiment run successfully I was very interested to rerun the other experiments that was the tasks plate for our group. Also, in the last class Prof Jonas had emphasized on using the copyExp.pl command to copy the experiments from last year’s directory to ours. I used the following command to copy the files:

  • copyExp.pl -t mnt/main/sp18/Exp0305/11 mnt/main/Exp/0314/002

I was able to copy the files successfully using the command however when I tried to run runAll.pl command at the end of decoding it gave me some error and it didn’t run. To resolve this issue, I met with Prof Jonas and we learnt together that there was a bug in copyExp.pl. What was basically happening was that if you have sp18 to the path it does not work however if you don't add that it does work fine. Prof Jonas and I created two experiments on the Caesar under in directory 0314/000 which has two experiments 'copied-from-in-Exp-only' and 'copied-from-in-sp18-in-Exp' and the two different experiments live there. The goal of creating these two experiments was to be able to distinguish why the train was failing before. This helped us learn that when sp18 is in the path the script doesn’t work and when it isn’t it works.

We also learnt something very weird. The time on Caesar was off by 4 hours and the time on the drones was off by 30 minutes which was why I was not able to get a time stamp. Decided to inform the systems group about the same.

02/08/2019 – I informed the experiments group about the bug in copyExp.pl and directed them to the directory where the two experiments lived and what Prof Jonas and I figured in my meeting with him on Thursday. Informed Don from the systems about the difference in the time on both Caesar and on the drones. After everything, I tried to decode the experiments that I worked on with Professor Jonas on Thursday and after running the following command:

      • sclite -r 002_train.trans -h hyp.trans -i swb >> scoring.log
        

I got an erroe which said “segmentation fault(core dumped)” I googled the error and learnt that I was trying to access the memory that I am not supposed to. I also reached out to Nick if he knew what the error meant. I finally emailed Prof Jonas and he suggested to look at my train.trans and hyp.trans files. After looking at the two files I realized that they were not the same hyp.trans was empty which suggests that parseDecode.pl script did not run.

I also did some reading about corpus and Sphinx.

02/11/2019 – I am still in the process of figuring this out. Our group met today to work on the proposal. I was not able to make to the meeting due to an emergency.


Results
Mentioned in the above section


Plan
For this coming week I plan to finish the 5 hour train on all the experiments assigned to the data group from the last year's data group.


Concerns
Mentioned in the above section

Week Ending February 18, 2019

Task

02/13 - Met with my group in school to work on the project and discuss the plan moving forward. We also wanted to discuss about how to proceed with the proposal that mostly Peter is working on for our group. Worked with Monica to successfully copy everything over from 0305/013 to 0314/005(These are the experiments from the Spring 2018 group that Monica decided to work on. She was not able to copy the experiment using copyExp.pl command. I fixed that following Professor Jonas's instructions from my one to one meeting with him last week. We opened sphinx_decode.cfg and sphinx_train.cfg using nano and changed the file paths to be 0314/005 instead of 0305/013. This worked fine, but we are sill having issues with the hype.trans because the parseDecode.pl didn't run properly and created and empty trans file. So, when we run the following command

  • sclite -r 002_train.trans -h hyp.trans -i swb >> scoring.log

It says segmentation fault(core dumped) This was the same error that I got when I ran this command on my 5 hour train. While trying to trouble shoot we looked at the hyp.trans-OLD file and it was not empty like hyp.trans. I also reached out to Nick from the experiments team to check if there was some bug with parseDecode.pl script and in order to check it, he ran the same command and informed us that he had the same issue.

02/18 - Still stuck at the segmentation fault error. Tried to figure out why the decode command didn't work properly. I checked the decode.log file as Professor Jonas suggested and it seemed like it did not run properly. I used the cat decode.log command to check the decode.log. I noticed the following error in the file: FATAL_ERROR: "mdef.c", line 680: No mdef-file AND ERROR: "cmd_ln.c", line 724: Cannot open configuration file /mnt/main/Exp/0314/002/model_parameters/002.cd_cont_100/feat.params for reading I am assuming this is because of the empty hyp.trans file which was created when parseDecode.pl ran.

Results
Mentioned in the above section
Plan
Fix the errors that Monica and I are running into when we run the following command:
  • sclite -r 002_train.trans -h hyp.trans -i swb >> scoring.log

Also, figure out why parseDecode.pl is creating an empty hyp.trans file.

Concerns
Get the right hyp.trans file.

Week Ending February 25, 2019

Task

During the in class meeting Professor Jonas directed us towards the direction as to how to resolve the segmentation fault (core dumped) error that we were getting. He suggested us to look at the top of decode.log, I used the cat decode.log to look at the decode.log and then ran the following command:

usr/local/bin/sphinx3_decode \
	-hmm /mnt/main/Exp/0314/002/model_parameters/002.cd_cont_100 \
	-lm /mnt/main/Exp/0314/002/LM/tmp.arpa \
	-dict /mnt/main/Exp/0314/002/etc/002.dic \
	-fdict /mnt/main/Exp/0314/002/etc/002.filler \
	-ctl /mnt/main/Exp/0314/002/etc/002_decode.fileids \
	-cepdir /mnt/main/Exp/0314/002/feat \
	-cepext .mfc 

This was basically what the decoder does. After running the command I got a bunch of stuff but, the most informative error message was the following:

ERROR: "cmd_ln.c", line 724: Cannot open configuration file /mnt/main/Exp/0314/002/model_parameters/002.cd_cont_100/feat.params for reading

Looking at this error, I went into model_parameters and I was able to figure out as to how to resolve that error. The issue was with 002.cd_cont_100, where the 002.cd_cont_100 should actually be 002.cd_cont_1000. My next course of action to resolve this issue was to run the following command:

nohup run_decode.pl 0314/002 0314/002 1000 &

After this I ran the parseDecode.pl decode.log hyp.trans which creates the hyp.trans file. Then, I used nano hyp.trans to see if the hyp.trans file was still empty or the error had been fixed and it actually had stuff in it, I realized that it was not empty anymore. Changing 100 to 1000 fixed the error and populated the hyp.trans file due to which the command to get the scoring log was throwing an Segmentation fault error. After this I ran the following command:

sclite -r 002_train.trans -h hyp.trans -i swb >> scoring.log

This was supposed to run fine because it ran fine for Monica and she and I followed the same steps but, somehow it failed for me. It gave me the following error:

Error: Not enough Reference files loaded

I looked up in the instructions for running a decode and this seemed to be one of the common errors I tried to follow the steps mentioned in the notes to debug it, but I have not had success so far. From the notes I learnt that this error is usually caused by duplicate identical transcript entries in either the hypothesis transcript and/or the reference transcript. It is mostly the hypothesis transcript that causes the error, so it is a best practice to focus on that. The steps consisted of the following: Remove all redundant lines.

    • We use a built-in Unix tool called uniq to do this for us. The output of this tool needs to go to a new file.
    • % uniq hyp.trans >> hyp.trans.uniq
  1. Restart SCLite while using the newly created hyp.trans.uniq file.
    • sclite -r <experiment #>_train.trans -h hyp.trans.uniq -i swb >> scoring.log

The steps mentioned above are the steps mentioned in the notes from last year. I followed these steps but, I am still getting the same error. Another thing that I noticed was when I nano into scoring.log I see the following table:


     |                            hyp.trans				                                    |
     |-----------------------------------------------------------------|
     | SPKR    | # Snt # Wrd | Corr    Sub    Del    Ins    Err  S.Err |
     |---------+-------------+-----------------------------------------|
     | sw2001b |    1      3 | 66.7   33.3    0.0   66.7  100.0  100.0 |
     |---------+-------------+-----------------------------------------|
     | sw2005a |    2     42 | 88.1    7.1    4.8    7.1   19.0  100.0 |
     |---------+-------------+-----------------------------------------|
     | sw2006b |    1     29 | 75.9   13.8   10.3    0.0   24.1  100.0 |

From my understanding of the table, the result is not good as the Err is 100. For Monica's experiment this was around 34.7 I guess, I plan to work on getting this right with her and my team mates. I also plan to run a 5 hr train on the other experiment from last year and then move on with the 30 hr train on these two experiments. However, I am a little confused about how the 30hr train would work, will it be exactly the same. I know it would take longer but, as far as results and the tasks are concerned, all the efforts up till now in being able to successfully run the 5 hr were to ultimately be able to run the 30 hr on these previous year's experiment. I know Monica worked on some database stuff and created some queries. I plan to touch base with her on that. I am also a little concerned about fixing the missing files error that I am running into.

Results
Mentioned above


Plan
Mentioned above


Concerns
Mentioned above

Week Ending March 4, 2019

Task

The 5hr experiment from last year (0305/011), is not working from me. I tried to redo it, I copied everything from the last year’s folder which is (0305/011) to my current folder and tried to repeat the steps and get everything to run to figure out what I did wrong what I actually messed up. After copying the files, I also went in to the file sphinx_decode.cfg and sphinx_decode.cfg and make the changes we made to the files by changing the file path as we had done before for Monica and as Professor Jonas suggested in my meeting with him. For example, in my case I made the following changes 0314/002 instead of 0305/012.

As planned, I also ran the 5hour on 0305/012 which is in 0314/003. I was responsible for 012 and 013 from last year. On this experiment I got success and got an error rate of 66.7% which as per my understanding is not accurate. I got the following log table after I ran the command for the scoring log. When I nano into scoring log I see the following table:

,-----------------------------------------------------------------.
      |                            hyp.trans				|
      |-----------------------------------------------------------------|
      | SPKR    | # Snt # Wrd | Corr    Sub    Del    Ins    Err  S.Err |
      |---------+-------------+-----------------------------------------|
      | sw2001b |    1      3 |100.0    0.0    0.0   66.7   66.7  100.0 |
      |---------+-------------+-----------------------------------------|
      | sw2005a |    2     42 | 76.2   14.3    9.5    4.8   28.6  100.0 |
      |---------+-------------+-----------------------------------------|
      | sw2006b |    1     29 | 69.0   20.7   10.3    3.4   34.5  100.0 |
      |---------+-------------+-----------------------------------------|

I also looked at the logs from last year for this particular experiment and learnt that the result I got was way different from last year’s experiment that Professor Jonas ran which was a standard 5hour train to make sure that [] and - marked words are in the language model, transcripts and dictionary.

Professor Jonas and Monica had an email exchange about the next steps moving forward after we have finished working on the 5hr trains we plan to move on to the 30hr train which is the most critical part of the all the tasks for the data group because the modeling group is heavily dependent on the data group for the quality of data that we provide them with to create the accurate models and therefore for the success of this entire project. From his email I inferred that moving forward we have to run a new 5-hour baseline that decodes on all 4172 (i.e. 5 hours) wave files to redo 0305/012 since it looks as if genTrans.pland pruneDictionary.pl has changed from when I ran 0305/012 last year. I would imagine 0305/011 and 0305/013 are still valid since they didn't rely on the existing genTrans.pl or pruneDictionary.pl (but rather modified versions that I created and stored in their respective /etc/scripts directories). Our next focus on the following files:

  • 0314/008 - baseline of how things work now (i.e. keep both [] and - designated annotation)
  • 0305/011 - remove both [] but keep - designated annotation
  • 0305/013 - remove both [] and - designated annotations.

I am concerned about running the 30hr as it requires a lot of digging in to the genTrans.pl and pruneDictionary.pl and we will have to alter and run these two files by hand and we cannot run them by a script. Monica and I plan on working on these two together and may be have Peter help us as well.



Results


Plan


Concerns

Week Ending March 11 and March18 , 2019

Task

During our last class meeting before the Spring break I had to leave early due to a family emergency. Professor Jonas informed me later in the evening that teams were created during the class time and I have been assigned to Alliance team with George, Christian, Peter, Ethan, Brooke, Scott, Wesley and Adam. Nick created a Slack channel for our group. We have not discussed about our plan moving forward as a group. I reached out to my group to check with them about what I had missed in the last class for the group. I am hoping that we would discuss in more detail about the plan moving forward as a group. Over the last weekend Monica and I discussed the plan for the Data team moving forward and she mentioned to me that Professor Jonas as assigned me, Brandon and Monica for the working on 0305/011 and 0305/013 30hr exps. I, Monica and Brandon worked together on analyzing the effects of bracketed and hyphened annotations. I also went over the big email Professor sent to the group and tried to learn about the filler words found in the experiment 0314/010. I also, read about how to create a corpus, I had seen Brandon work on it while he walked all of us through it but, I had personally never done it. I went through the notes from last year and did some more research about the entire purpose of a corpus and how is it used in the professional world. After doing some research I learnt that a speech corpus is basically a database of speech audio files and text transcriptions. I also learnt that in speech technology, speech corpora are used among other things to create acoustic models (which can then be used with a speech recognition engine). We had already discussed about this in class a couple of times but, I wanted to have a better understanding of this before starting off with the 30 hr. I know Monica did like a simple 30hr experiment and it gave her a 50.7% WER. I will be looking at that with her tomorrow and will start to catch up from this week because I feel I have been a little behind during the last two weeks. Also, I ran the 5 hr experiment on 0314/003 last week in class and it worked and I got the following:

				
			SYSTEM SUMMARY PERCENTAGES by SPEAKER

      ,-----------------------------------------------------------------.
      |                            hyp.trans				|
      |-----------------------------------------------------------------|
      | SPKR    | # Snt # Wrd | Corr    Sub    Del    Ins    Err  S.Err |
      |---------+-------------+-----------------------------------------|
      | sw2001b |    1      3 |100.0    0.0    0.0   66.7   66.7  100.0 |
      |---------+-------------+-----------------------------------------|
      | sw2005a |    2     42 | 76.2   14.3    9.5    4.8   28.6  100.0 |
      |---------+-------------+-----------------------------------------|
      | sw2006b |    1     29 | 69.0   20.7   10.3    3.4   34.5  100.0 |
      |---------+-------------+-----------------------------------------|

My major concerns moving forward are the 30 hr experiments that we three have to work on. I know it will be a good learning curve but, as Professor Jonas mentioned it is a tedious task because we manually have to go in and work with the Perl Scripts. I do have some experience working with Perl from my programming languages class that I took in last fall. We did cover some basic perl but, I am excited and nervous at the same time to work perl scripts again. My another concern, a short term one is to get the simple 30hr experiment going on my own. I know Monica did it but, I want to do it on my own once to have a better understanding.



Results


Plan


Concerns

Week Ending March 25, 2019

Task

This week in class Professor Jonas sort of combined the data group and the modeling group together because he really wanted the 30hr experiments from the summer 18 to be rerun. I teamed up with George to repeat the three experiments with both 30hr and 145 hour train. We were also required to be careful while using the correct senone count, mixture weight and the same random seed number as it was used in the summer 2018 experiments by Hannah. This task involved some research and a lot of email exchanges which George did to figure out the best senone count and mixture weights for the 30 and 145hr experiments. Another thing that required some research was to figure out where the changes were to be made in the files.

George and I decided to split the work as he had most of the research about what changes needs to be done to what files. He decided to take care of the 30hr experiment for the three experiments with best WER from Summer 2018. I am responsible for running the 145 hr. George also helped me with all the configurations for one 145hr which lives in 0316/012 which is still running and I plan to leave it running over night and then move on to the next steps tomorrow. We did a couple of changes to some of the files in order to change to configuration and for the train to run perfectly fine for 145hr experiments. These are the following changes that were made to the files for 145 train.

etc/sphinx_train.cfg:
$CFG_HMM_TYPE = '.cont.'
$CFG_LDA_MLLT = 'yes'
$CFG_LDA_DIMENSION = 32
$CFG_FINAL_NUM_DENSITIES = 32
$CFG_N_TIED_STATES = 8000
python/sphinx/mllt.py:
r = 2038113770 

To have a better understanding of what we were actually supposed to do for the tasks assigned for this week I decided to go back and read the logs of last year’s Modeling group. They were very self explanatory and I felt I could follow it and it also helped me understand the tasks even better. George also explained me what is exactly expected of us. We have been communicating via Slack to coordinate and work together.

On Monday morning George finished the 30hr train and we planned to have a Zoom meeting Monday night to discuss about the 145hr train. When we first ran the trains last week we didn’t realize that we were not using the right mixture weights. We did some brain storming on that and figured what was the right mixture weights.

As of now we plan to finish the decoding and training the 30 and 145hr experiments and wait for the best WER. We plan to have these two tasks done before tomorrow’s class meeting on 03/26 or by tomorrow midnight.

March 26, 2019 - The last time I checked the train for 0316/012 was still running. I kept it running last night and I am waiting for that to finish and then I can move ahead with the process ahead.


Results


Plan


Concerns
Right now our main concerns are getting the trains done in a timely manner so, that we have something solid for the class meeting tomorrow. I know George is pretty close to being done with the decode for his 30hr experiment.

I will also be updating my log tomorrow with the results that we get from the train that we ran tonight.

Week Ending April 1, 2019

Task

Last week in class we discussed steps and roles moving forward. As I mentioned in my last log Prof combined the data group with the modelling group to perform some of the crucial tasks. Also, mentioned in my last log George and I were supposed to be working on the rerun of the experiments with best WER from summer of 2018. George re ran a 30hr train for the top three experiments which were 0312/006, 0312/012 and 0312/013. On the other hand I was responsible for running the 145hr train and then building the language model as well as decoding it.

Until last class the 145hr train for 013/12 was not completed. It took around 20hrs to train that. After the train was done we built a language model and then ran the decode using the following command:


/usr/local/bin/sphinx3_decode -lda /mnt/main/Exp/0316/012/model_parameters/012.mllt_cd_cont_8000/feature_transform -hmm /mnt/main/Exp/0316/012/model_parameters/012.mllt_cd_cont_8000 -lm /mnt/main/Exp/0316/012/LM/tmp.arpa -dict /mnt/main/Exp/0316/012/etc/012.dic -fdict /mnt/main/Exp/0316/012/etc/012.filler -ctl /mnt/main/Exp/0316/012/etc/012_decode.fileids -cepdir /mnt/main/Exp/0316/012/feat -cepext .mfc >& decode.log &

While the decode for this was running we fired up the train for 0312/012 and 0312/013. The decode took almost 14 15 hours to run while the trin again took longer. Due to some reason my computer was slower than George’s so, George ran the train for 0312/013. After we built the language model and ran the decode, we then wanted to create a scoring log to get the WER to see how the senone count, longer train and increase of mixture weights impacted the WER. After, obtaining the scoring log for this experiment we observed the result to be worse than what we got for 30hr. Here is what we got for 0312/006 which is in 0316/012


     | Sum/Avg | 3898  57574 | 80.6   11.0    8.4    2.9    22.3    78.2 |
     |===================================================================|
     |  Mean   |  1.3   18.7 | 81.2   11.6    7.1    6.7    25.5    78.4 |
     |  S.D.   |  0.5   15.9 | 16.6   12.4   10.0   30.8    34.5    38.6 |
     | Median  |  1.0   14.0 | 83.8    8.7    3.2    0.0    20.5   100.0 |
     `-------------------------------------------------------------------'     

During the last class meeting we also met our teams. I am on team Alliances and we discussed about our steps moving forward and wht we would like to achieve as a team. Prof Jonas gave us some interesting ideas and we all decided to do some of the research to come up with a good plan this week in class.

The scoring log for the 0312/012 which is 0316/013

     | sw4940b |    2     13 | 61.5   15.4   23.1    0.0    38.5    50.0 |
     |===================================================================|
     | Sum/Avg | 3898  57574 | 77.7   12.9    9.4    3.3    25.6    81.1 |
     |===================================================================|
     |  Mean   |  1.3   18.7 | 78.9   13.4    7.7    7.3    28.5    81.5 |
     |  S.D.   |  0.5   15.9 | 17.3   13.2   10.2   30.1    34.0    36.3 |
     | Median  |  1.0   14.0 | 80.6   11.1    4.2    0.0    25.0   100.0 |
     `-------------------------------------------------------------------'

This WER is worse in comparison to the 30hr experiment for the same experiment.

The scoring log for the 0312/013 which is 0316/014

     | sw4940b |    2     13 | 61.5   15.4   23.1    0.0    38.5    50.0 |
     |===================================================================|
     | Sum/Avg | 3898  57574 | 77.7   12.9    9.4    3.3    25.6    81.1 |
     |===================================================================|
     |  Mean   |  1.3   18.7 | 78.9   13.4    7.7    7.3    28.5    81.5 |
     |  S.D.   |  0.5   15.9 | 17.3   13.2   10.2   30.1    34.0    36.3 |
     | Median  |  1.0   14.0 | 80.6   11.1    4.2    0.0    25.0   100.0 |
     `-------------------------------------------------------------------'


Results


Plan


Concerns

Week Ending April 8, 2019

Task


Results


Plan


Concerns


Week Ending April 15, 2019

Task


Results


Plan


Concerns


Week Ending April 22, 2019

Task


Results


Plan


Concerns


Week Ending April 29, 2019

Task


Results


Plan


Concerns


Week Ending May 6, 2019

Task


Results


Plan


Concerns