/export/mt/ws99/tools/bin/corpus_analyze.prl run by on Mon Aug 2 20:41:44 1999 Analysis of rd_plain_norm-train.snt: ================================================================================ CORPUS DATA: rd_plain_norm-train.snt contains 44538 sentences (44450 unique). -------------------------------------------------------------------------------- TOKENS TYPES (C > 1)* -------------------------------------------------------------------------------- SOURCE: rd_plain_norm-train.snt 695935 30949 (18876) TARGET: rd_plain_norm-train.snt 616872 71944 (33245) -------------------------------------------------------------------------------- *The number of types which appear more than once. ================================================================================ CO-OCCURENCE DATA: -------------------------------------------------------------------------------- TOTAL UNIQUE (C > 1)* -------------------------------------------------------------------------------- rd_plain_norm-train.snt 11265867 4354395 (948628) -------------------------------------------------------------------------------- *The number of co-occurences which appear more than once. ================================================================================ SENTENCE LENGTH DATA: -------------------------------------------------------------------------------- rd_plain_norm-train.snt SOURCE: AVERAGE SENTENCE LENGTH: 15.6256455161884 LENGTH COUNT (* = 39 SENTENCES) 1 : 7 2 : 225 ***** 3 : 617 *************** 4 : 764 ******************* 5 : 1019 ************************** 6 : 1404 ************************************ 7 : 1593 **************************************** 8 : 1717 ******************************************** 9 : 1928 ************************************************* 10 : 2153 ******************************************************* 11 : 2247 ********************************************************* 12 : 2253 ********************************************************* 13 : 2252 ********************************************************* 14 : 2274 ********************************************************** 15 : 2284 ********************************************************** 16 : 2262 ********************************************************** 17 : 2174 ******************************************************* 18 : 2065 **************************************************** 19 : 1956 ************************************************** 20 : 1744 ******************************************** 21 : 1700 ******************************************* 22 : 1551 *************************************** 23 : 1454 ************************************* 24 : 1337 ********************************** 25 : 1155 ***************************** 26 : 1133 ***************************** 27 : 969 ************************ 28 : 852 ********************* 29 : 740 ****************** 30 : 621 *************** TARGET: AVERAGE SENTENCE LENGTH: 13.8504647716557 LENGTH COUNT (* = 39 SENTENCES) 1 : 14 2 : 276 ******* 3 : 739 ****************** 4 : 1085 *************************** 5 : 1522 *************************************** 6 : 1894 ************************************************ 7 : 2175 ******************************************************* 8 : 2349 ************************************************************ 9 : 2524 **************************************************************** 10 : 2522 **************************************************************** 11 : 2564 ***************************************************************** 12 : 2587 ****************************************************************** 13 : 2564 ***************************************************************** 14 : 2423 ************************************************************** 15 : 2365 ************************************************************ 16 : 2193 ******************************************************** 17 : 1997 *************************************************** 18 : 1935 ************************************************* 19 : 1675 ****************************************** 20 : 1550 *************************************** 21 : 1397 *********************************** 22 : 1278 ******************************** 23 : 1065 *************************** 24 : 858 ********************** 25 : 813 ******************** 26 : 655 **************** 27 : 534 ************* 28 : 386 ********* 29 : 298 ******* 30 : 213 ***** ================================================================================