Speech:Spring 2016 Daisuke Matsukura Log


 * Home
 * Semesters
 * Spring 2016
 * Proposal
 * Report
 * Information - General Project Information
 * Experiments - List of speech experiments

Week Ending February 9, 2016

 * Task:
 * 2/3 Read logs and documentation to understand what we supposed to do in this semester. Exchange information of the teammate & say hello the. Login as a root to the Caesar Server.
 * 2/4 Try to understand specific part, at this moment accessing to Caesar Server internally from putty in the school (internal environment). Read logs and documentation same as 2/3.
 * 2/5 Read logs and documentation same as 2/4.
 * 2/6 Try to install RedHat Enterprise 6.6 Linux in my environment. Continue to read document. Exchange some information with team and asked to professor about external connection.
 * 2/9 Retry to connect from house to Caesar.


 * Results:
 * 2/3 I did understand some hierarchy of the directories and Unix commands. Connect from internal environment using putty to Caesar server.
 * 2/4 Become understand some documentation and this all team's task purpose.
 * 2/5 I understand some point importance to concentration in the project for tools team. Current directory setup hierarchy.
 * 2/6 Successfully installed and configure basic setting internally.
 * 2/9 Successfully connected as a root to Caesar. It is passing unh Pulse Secure and I used putty to connect. However somehow I was not able to connect using SSH Secure Shell Client.


 * Plan:
 * 2/3 Try to understand more detail.
 * 2/4 Same as 2/3
 * 2/5 Same as 2/4
 * 2/6 It needs configure access from internal network and outside of the network.
 * 2/9 Still working on to establish connection.


 * Concerns:
 * 2/3 I need better to understand more detail. In order to do that, I have to read carefully and understand as much as possible, which is tough for me at some point and it is make me kind of tricky.
 * 2/4 Same as 2/3
 * 2/5 Same as 2/3
 * 2/6 Connection to the server. Which does not work for me from external environment.
 * 2/9 I easily misundestand about what they write. And I really need read carefully.

Week Ending February 16, 2016

 * Task:
 * 2/10 Login as a my user name and change password.
 * 2/11 Explore and try to use discussion function of this course IT ware in order to share information in the team.
 * 2/14 Read tools part of the sphinx decorder article and try to use Sphinx decorder on my environment as a test purpose.


 * Results:
 * 2/10 Successfully changed password from root. And then I could login on my account on caesar.

[root@caesar ~]# find /var -user dm**** /var/spool/mail/dm1097 [root@caesar ~]# passwd dm**** Changing password for user dm****. New password: ******** Retype new password: ******** passwd: all authentication tokens updated successfully. [root@caesar ~]#


 * 2/11 I found posting and share and each team member can share the information in this tool. But basic this discussion function needs improve by system.
 * 2/14 It still working on the configuration part. Sphinx is ready to install.


 * Plan:
 * 2/10 Explorer more and concentrate to understand about setting up tools.
 * 2/11 Try to use more for share with group member. Start to read Sphinx documentation and try to use on my environment.
 * 2/14 Install actuall sphinx on my environment and understand how to use and storucture of the tool.


 * Concerns:
 * 2/10 Same as 2/9. In addition, we still have concerns about which part is our portion of the task as a tool teams.
 * 2/11 Same as 2/10.
 * 2/14 Same as 2/11.

Week Ending February 23, 2016
2/17 Task
 * Task:
 * Find version of the sphinx installed or not.
 * And which version is in.
 * Add include finish proposal.
 * Take a look SCLITE.
 * Meeting on Sunday 8pm.
 * Write a proposal as a tools group.

2/17 Did not find installed Sphinx so far. I do not know is that spelling miss or name of the soft name is wrong.
 * Results:

2/22 Read through SCLITE documentation and understand some feature. SCLITE is a part of the NIST Scoreing Toolkit (SCTK). And it is a collection of software tools designed to score benchmark test evaluations of Automatic Speech Recognition(ASR)Systems. It includes three programs, sclite(SCLITE), sc_stats, and rover.

2/23 Sam as 22 and write proposal. As mentioned tools group's proposal, I will do;


 * Make manual of wiki discussion page use in the group 
 * Research SCLITE, check difference current and newest version on the server and if need to be update install newest version. "Software Implementation" wiki page accordingly
 * Research, Sphinx base and CMU Cam Toolkit. Update "Software Implementation" wiki page accordingly
 * Install if these need to be and document on wiki screen usage

2/17 Research more directory and access to mnt/main/local directory and explorer inside of the directory. Especially this time, I tried to concentrate on the installing file. As well as try to understand structure of the directory.
 * Plan:

2/17 I still wonder what we supposed to do towards proposal. But I try to find more specific detail of the SCLLITE as much I can.
 * Concerns:

2/23 Now, I start understand structure of SCLITE and it has a part of the toolkit of NIST SCTK Scoring Toolkit. But I still have to understand the actual function and how to use it.

Week Ending March 1, 2016
2/26 Continue to read documentation SCLite and check its version. 2/29 Start to create project timeline. 3/2 Test to upload project timeline image.
 * Task:

2/29 Find the SCLite version is, sclite Version: 2.3, SCTK Version: 1.3 And now it is available on, sctk-2.4.10-20151007-1312Z.tar.bz2 (MD5).
 * Results:

This is newer version than Speech System Information, SCLITE 2.9(included in Speech Recognition Scoring Toolkit (SCTK) Version 2.4.8)

3/1 Finished up to first draft of class project timeline and send to all of the classmate by email.

3/2 Successfully uploaded project timeline in bellow(first draft file).



2/29 Discuss with group about current version of SCLITE and update process. It may need test first and compare contrast version, which gives accurate result newer or old version.
 * Plan:

3/1 Fix time schedule, when I get response from all of the group's comment or modify request.

3/2 Talk with all group about time line, exchange each group member's progress and talk about next task in the class. 2/29 Still I have to read documents and try to do it. For me it takes time to do.
 * Concerns:

3/1 Same as 2/29

3/2 Same as 3/1

Week Ending March 8, 2016
3/2
 * Task:
 * Find installed application's file(include backup file) location.
 * Update Speech: Function page of SCLITE on Speech System Information table.

3/2 Back up application file location is in,
 * Results:

[dm1097@caesar tar]$ ls -al total 323668 drwxrwxr-x. 2 root root    4096 Jun 27  2012. drwxrwxr-x. 4 root root    4096 Jun 27  2012 .. -rw-rw-r--. 1 root root 64327561 Jun 27 2012 an4_sphere.tar.gz -rw-rw-r--. 1 root root 25321975 Jun 27 2012 cmusphinx-sphinx3.tar.gz -rw-rw-r--. 1 root root 18271146 Jun 27 2012 dropbox.tar.gz -rw-rw-r--. 1 root root 66328720 Jun 27 2012 gcc-4.5.2.tar.bz2 -rw-rw-r--. 1 root root 19517440 Jun 27 2012 media.tar -rw-rw-r--. 1 root root 20398080 Jun 27 2012 redmine-1.1.1.tar -rw-rw-r--. 1 root root   30720 Jun 27  2012 sclite.tar -rw-rw-r--. 1 root root 14100480 Jun 27 2012 sctk-2.4.0-20091110-0958.tar -rw-rw-r--. 1 root root 46991360 Jun 27 2012 speechtools.tar -rw-rw-r--. 1 root root 50305024 Jun 27 2012 sphinx3.tar -rw-rw-r--. 1 root root 5819368 Jun 27  2012 torque.tar.gz [dm1097@caesar tar]$ pwd /mnt/main/backup/System/install/tar

Install application file location is in, [dm1097@caesar main]$ ls backup Exp   install  lost+found  rsyncBackups corpus home  local    misc        scripts ;Plan: [dm1097@caesar tar]$ ls an4_sphere.tar.gz        sctk-2.4.0-20091110-0958.tar.gz cmusphinx-sphinx3.tar.gz  speechtools.tar.gz dropbox.tar.gz            sphinx3-0.8.tar.bz2 gcc-4.5.2.tar.bz2        sphinx3.tar.gz media.tar.gz              sphinxbase-0.7.tar.gz redmine-1.1.1.tar.gz      torque-3.0.0.tar.gz sclite.tar.gz             torque-4.2.3.1.tar.gz [dm1097@caesar tar]$ pwd /mnt/main/install/tar Installed application location is in, [dm1097@caesar bin]$ ls binlm2arpa     printserverdb  qsub                sphinx3_livepretend chk_tree       printtracking  qterm               sphinx3_lm_convert ctags          qalter         rcs-checkin         sphinx3-simple dropbox.py     qchkpt         rover               sphinx_cepview ebrowse        qdel           sclite              sphinx_cont_adseg etags          qdisable       sc_stats            sphinx_cont_fileseg evallm         qenable        sox                 sphinx_fe f2py           qhold          soxi                sphinx_jsgf2fsg grep-changelog qmgr           speech              sphinx_lm_convert hostn          qmove          sph2pipe            sphinx_lm_eval idngram2lm     qmsg           sphinx3_align       sphinx_lm_sort idngram2stats  qnodes         sphinx3_astar       sphinx_pitch interpolate    qorder         sphinx3_cfg2fsg     text2idngram mergeidngram   qrerun         sphinx3_conf        text2wfreq ngram2mgram    qrls           sphinx3_continuous  text2wngram nqs2pbs        qrun           sphinx3_dag         tracejob pbs-config     qselect        sphinx3_decode      w_edit pbsdsh         qsig           sphinx3_dp          wfreq2vocab pbsnodes       qstart         sphinx3_ep          wngram2idngram pbs_track      qstat          sphinx3_gausubvq printjob       qstop          sphinx3_livedecode [dm1097@caesar bin]$ pwd /mnt/main/local/bin

Updated on SCLITE on Speech System Information table


 * Plan;

Read thorough and understand what group member did so far and read article of my part.


 * Concerns:

It takes time to get thorough and really understand, what write. Especially I had difficulty to figure out accurate information from external site what is "Differences between current and newest versions" on the Speech System Information. Even these small things time consuming for me to understand.

As a team or class, I did some misunderstanding. So, when I notice it was late to fix such as time line. I was thinking that was important not only proposal but doing class's process. I would not do misunderstand, but it occur often.

Week Ending March 22, 2016
3/9 - 3/14
 * Task:
 * 1) Login to majestix and check current status.
 * 2) Read previous tools group as well as related log entries.
 * 3) Continue to read and understand software and its research.
 * 4) Read documents and discuss about our plan to set up on majestix.
 * 5) Read through and try to understand speech recognition experiment.
 * 6) Write and updated Tools team task

3/19 same as 3/9 - 3/14

3/20 same as 3/9 - 3/14

3/21 run train

3/9 Successfully run speech recognition application and reading through Run a Train Setup Script I got some idea and concept run a train.
 * Results:

3/13 update tools group's task for spring break period of week.

3/19 Tried to start speech experience to run train. But I still got confused running createWikiExperiment.pl as well as location of that perl script. And read through document again.

3/20 I could not find the reason and location of createWikiExperiment.pl perl script. So, I decided to create under wiki page Speech:Exps 0286 003 Author: Daisuke Matsukura Date: 3/22/2016 Purpose: Experiment used to train on 5hr Details: Secret Sauce A Results: And I read and try to understand how to run appropriately.

3/21

3/9 - 3/14 Read more document to get understand. 3/19 Same as 3/9 - 3/14. And try to start running train.
 * Plan:

3/20 Same as 3/19

3/21 Copy createWiki_Sub_Experiment.pl (Location: /mnt/main/scripts/user) to my user directory. copies the createWiki_Sub_Experiment.pl script to /mnt/main/scripts/user on cisunix under the my username.

3/9 I still have misunderstanding myself. I should be careful and not only read document, I notice that it is important to ask to group about unclear or question to them.
 * Concerns:

3/19 Just do it.

3/20 Just do it.

3/21 Just do it.

Week Ending March 29, 2016

 * Task:
 * 1) Update tools group tasks
 * 2) Search and install Screen on Majestix
 * 3) SSH seamless login into Ceaser from Majestix or other servers
 * 4) Learn how to run Train(continue)

3/23 Meeting with group. And set up next weeks task then write down to the. Research Linux command line software, screen and test install on my own environment and successfully installed it. Asked to professor Jonas about location. By default location he assumed to be install in /use/local. Because of the point if the installation is to know what files are installed where so we can do it by hand on caesar eventually. So, we may want to take a snapshot of each directory that might be changed, i.e. /bin, /sbin, /share/bin, /sys/bin, /usr/bin, /lib, /share/lib, /usr/lib, /sys/lib.
 * Results:

3/24 Continue to work to install. Research from website.

3/25 Continue to work to install. Research from website.

3/26 Continue to work to install. Research from website and wikipage log. Ask question to professor Jonas to get clarify the way of making screen shot in order to install.

And this is the way of taking snapshot.

3/23 Install multiple command line tool screen. Keep in track of changed application location using history file in case of get wrong. And I need search if I do have unknown part using webpage and wiki.
 * Plan:

3/24 Same as 3/23.

3/25 Same as 3/24.

3/26 Same as 3/27.

3/23 N/A
 * Concerns:

3/24 N/A

3/25 N/A

3/26 N/A

3/27 N/A

Week Ending April 5, 2016

 * Task:

3/30 Keep in touch Tools and System group and wait new process. Start to discussion captain america.

3/31 Same as 3/30

4/1 Same as 3/30

4/5 Research tool Screen


 * Results:

3/30 Tools group: wain Majestix will be available to install. Captain America: in class meeting. Checked each of team's task.

3/31 Tools group: started to install gcc, emacs. Contact with tools group. Research installation process and way of remove tool.

4/1 Tools group: Same as 3/30

4/5 Screen share Multiuser Session

These commands allow other users to gain access to one single screen session. When attaching to a multiuser screen the sessionname is specified as username/sessionname to the -S command line option. Screen must be compiled with multiuser support to enable features described here.

Multiuser: Enable / Disable multiuser mode. Acladd: Enable a specific user. Aclchg: Change a users permissions. Acldel: Disable a specific user. Aclgrp: Grant a user permissions to other users. Displays: List all active users at their displays. Umask: Predefine access to new windows. Wall: Write a message to all users. Writelock: Grant exclusive window access. Su: Substitute user.
 * Plan:

3/30 Once ready to install other tools, start to install screen. And keep in touch with groups. To share information and research issues.

3/31 same as 3/30

4/1 same as 3/30
 * Concerns:

3/30 In order to install successfully, I have to keep in mind backup first then install on Majestix with out any problem.

3/31 same as 3/30

4/1 same as 3/31

Week Ending April 12, 2016
4/6 - 4/11
 * Task:
 * 1) Talk about poster with tools group
 * 2) Make documentation way of transfer file between Caesar and Majestix.
 * 3) Create manual documentation of the screen.
 * 4) Try and run train for my individual experiment.


 * Results:

4/6
 * 1) Try to test transfer file between Ceasar and Majestix using SFTP or SCP. Conceptually SFTP stands for Secure(or SSH) File Transfer Protocol and it allows to data access and data transfer over Secure Shell(SSH) data stream. SCP stands for Secure Copy and it allows to transfer computer files between a local host and remote host or two remote host securely.

Make documentation of it and post on the Wikipage and share with tools group. So far, I found the way of connection using sftp. But I still did not able to pull the data from Ceasar to Majestix using SCP. The result was, [user@caesar ~/appsfile]$ scp user@caesar:mnt/main/home/sp16/user/appsfile/screen-4.3.1 user@majestix:tmp/toolsgroup-screen The authenticity of host 'caesar (192.168.10.1)' can't be established. RSA key fingerprint is *******. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'caesar,192.168.10.1' (RSA) to the list of known hosts. Could not chdir to home directory /home/sp16/dm1097: No such file or directory mnt/main/home/sp16/dm1097/appsfile/screen-4.3.1: No such file or directory Killed by signal 1. Connection to caesar closed.
 * 1) Still we need keep in touch about making poster, which due is next week. Basically each people make different portion. My case, I will make some tool's graphics./

4/7 Read more web document and try to do again SFTP connection. This time I tried to make connection from Majestix to Ceaser.

Fist, establish connection to Majestix using SSH, Once enter the Majestix, do sftp caesar

Then find the location previously downloaded and extracted file directory name, screen-4.3.1(this is located on /mnt/main/home/sp16/dm1097/appsfile

4/8 More research about SFTP.

4/9 Change the way of transfer method to send as compressed file, gz to Majestix. Still SFTP does not allow to send to Majestix.

4/10 Created Tools group's poster images.

4/11

4/12


 * Plan:


 * Concerns:

Week Ending April 19, 2016

 * Task:

4/13 - 4/19


 * 1) Create documentation of Screen and related things
 * 2) Help Jon's task to find gcc and emacs installation files location
 * 3) Documentation and research feature of the rSync and connection over the network to make back up
 * 4) Research Captain America team's assigned task


 * Results:

4/13 Creating documentation of Screen and related things may not be needed. because it may be near close to the feature of Emacs(but need research of sharing screen has on Ceaser??). Also I have to convince professor Jonas about its necessary point to install to Majestix. Using SFTP get command works and successfully copy from Ceaser to Majestix.

4/14 Research and creating documentation step by step.

4/15 Continue 4/14

4/17 Finish creating the documentation and notify to Tools group, How to transfer file using SFTP
 * Plan:

4/13 Keep creating document and search feature comparison between Emacs and Screen. Make SFTP documentation how to do it and temporally test file location.

4/14-19 Read documentation of rSync and Emacs.
 * Concerns:

Week Ending April 26, 2016
4/20 - 4/21
 * Task:
 * 1) Study about running and decoding train and see the difference method and that result.
 * 2) Create documentation of tools groups additional things if needs
 * 3) Help Jon's task to find gcc and emacs installation files location
 * 4) Documentation and research feature of the rSync and connection over the network to make back up
 * 5) Research Captain America team's assigned task

4/21 Group meeting and shared additional information of the running train.
 * Results:

4/22 Still continue to research the way of decoding.

4/24 Catch up Tools group's logs what each member's tasks such as Jon did so far about emacs. And start to look up the way of screen installation.

4/25 Read installation docs of screen.

4/21 Study about running and decoding train and see the difference method and that result.
 * Plan:

4/22 Same as 4/21.

4/24 Same as 4/22.

4/26 Same as 4/26. And start to install screen.
 * Concerns:

Week Ending May 3, 2016
4/27 - 5/3
 * Task:
 * 1) Try to do decode process in order to more understand.
 * 2) Install screen in Majestix.

4/27 Try to do Sphinx decode process and created 30 hrs on 008
 * Results:

4/28 Read tools group installation documentation and find installation of.

4/29 Same as 4/28.

5/1 Talk with Jon about installing screen on Majestix.

5/2 Continue 5/1. And created Screen Install Documentation log and that link is belongs Tools groups page as well.

5/3 Continue 5/2. Try to install different version of screen instead of 4.0.3. This is the screen version list. Tools group meeting using hangout. And great contribution from Tom and Jon, successfully install screen on Majestix.

4/27
 * Plan:

4/28

4/29 Install screen. And try to solve issue of first trial.

5/1 same as 4/29.

5/2 same as 4/29

5/3 Change and try to install again with different version of screen Third & Fourth trial.

4/27
 * Concerns:

4/28

4/29 Try to find solve Screen installation error, which is unknown and makes concern to solve about 'pty.c:38:26: error: sys/stropts.h: No such file or directory make: *** [pty.o] Error 1'.

5/1 same as 4/29.

5/2 Same as 5/1.

5/3 Same as 5/2.

Week Ending May 10, 2016

 * Task:

5/3 Create Screen instruction and complete screen installation documentation.

5/7 Write final report documentation.

5/8 Add and fix final report documentation.

5/9 Same as 5/8.

5/10 Same as 5/9.


 * Results:

5/3 Finished create documentation. But before submit, it may need some modification and add some detail.

5/7 Wrote some final report.

5/8 Same as 5/7. Add more report.

5/9 Same as 5/8. Fix some my document.

5/10 Same as 5/9. Fix my documentation.
 * Plan:

5/3 - 5/10 Create and fix documentation.


 * Concerns:

5/3 None.

5/7 None.

5/8 None.

5/9 None.

5/10 None.