Speech:Exps 0057

From Openitware
Jump to: navigation, search

Description

Author: Jake (Joseph) Gallagher on Majestix

Date: 3/29/2013

Purpose: Attempt to run a Mini Eval successfully followed by a decode and score.

Details: After attempting a Mini train previously and running into some errors which need to be sorted out I wanted to try a Mini eval since some of my group members had success running it. After setting up the experiment folder and respective files I ran a successful train. At the conclusion of the train I started the decode process and ran into some issues which are currently documented. I was able to troubleshoot and run the decode successfully. Errors encountered are listed below:

Error: double reference text for id '(sw2479a-ms98-a-0071)'
Error: Not enough Reference files loaded
Missing:
    (sw2466b-ms98-a-0071)
    (sw2470a-ms98-a-0014)
    (sw2479a-ms98-a-0071)
    (sw2493b-ms98-a-0059)
    (sw2534a-ms98-a-0098)
    (sw2547a-ms98-a-0059)
    (sw2547b-ms98-a-0099)

When starting the scoring I ran into an issue where the hyp.trans file was not actually being created in the etc folder so the scoring script would not successfully complete the scoring by creating a scoring.log. Instead of having the script make the file in the etc folder I opted to have it created in the DECODE folder and then moved it to the etc folder. This solved my problem and the scoring completed without issue.


Results: Successful Mini Eval test completed with scoring results:

                   SYSTEM SUMMARY PERCENTAGES by SPEAKER                      

      ,-----------------------------------------------------------------.
      |                         hyp.trans.uniq                          |
      |-----------------------------------------------------------------|
      | SPKR    | # Snt # Wrd | Corr    Sub    Del    Ins    Err  S.Err |
      |=================================================================|
      | Sum/Avg |  518  11051 | 80.7   11.2    8.0    7.6   26.8   91.1 |
      |=================================================================|
      |  Mean   |  2.7   58.2 | 80.8   11.8    7.4   12.2   31.4   92.9 |
      |  S.D.   |  1.7   43.0 | 12.5    8.7    8.4   17.8   20.3   17.8 |
      | Median  |  2.0   52.5 | 82.4   10.2    5.9    6.5   27.2  100.0 |
      `-----------------------------------------------------------------'