../../../tools/bin/corpus_analyze.prl run by knight on Mon Aug 2 15:19:01 1999 Analysis of experiments/50k/50k-train.snt: ================================================================================ CORPUS DATA: experiments/50k/50k-train.snt contains 43750 sentences (43662 unique). -------------------------------------------------------------------------------- TOKENS TYPES (C > 1)* -------------------------------------------------------------------------------- SOURCE: /50k/50k-train.snt 672963 30447 (18515) TARGET: /50k/50k-train.snt 597181 70488 (32441) -------------------------------------------------------------------------------- *The number of types which appear more than once. ================================================================================ CO-OCCURENCE DATA: -------------------------------------------------------------------------------- TOTAL UNIQUE (C > 1)* -------------------------------------------------------------------------------- /50k/50k-train.snt 10695214 4160581 (902129) -------------------------------------------------------------------------------- *The number of co-occurences which appear more than once. ================================================================================ SENTENCE LENGTH DATA: -------------------------------------------------------------------------------- experiments/50k/50k-train.snt SOURCE: AVERAGE SENTENCE LENGTH: 15.3820114285714 LENGTH COUNT (* = 39 SENTENCES) 1 : 7 2 : 225 ***** 3 : 617 *************** 4 : 764 ******************* 5 : 1019 ************************** 6 : 1404 ************************************ 7 : 1593 **************************************** 8 : 1717 ******************************************** 9 : 1928 ************************************************* 10 : 2152 ******************************************************* 11 : 2247 ********************************************************* 12 : 2253 ********************************************************* 13 : 2252 ********************************************************* 14 : 2274 ********************************************************** 15 : 2282 ********************************************************** 16 : 2261 ********************************************************* 17 : 2173 ******************************************************* 18 : 2065 **************************************************** 19 : 1952 ************************************************** 20 : 1739 ******************************************** 21 : 1697 ******************************************* 22 : 1546 *************************************** 23 : 1448 ************************************* 24 : 1329 ********************************** 25 : 1133 ***************************** 26 : 1119 **************************** 27 : 944 ************************ 28 : 823 ********************* 29 : 699 ***************** TARGET: AVERAGE SENTENCE LENGTH: 13.6498514285714 LENGTH COUNT (* = 39 SENTENCES) 1 : 14 2 : 275 ******* 3 : 739 ****************** 4 : 1083 *************************** 5 : 1522 *************************************** 6 : 1893 ************************************************ 7 : 2174 ******************************************************* 8 : 2348 ************************************************************ 9 : 2523 **************************************************************** 10 : 2520 **************************************************************** 11 : 2561 ***************************************************************** 12 : 2584 ****************************************************************** 13 : 2561 ***************************************************************** 14 : 2417 ************************************************************* 15 : 2354 ************************************************************ 16 : 2179 ******************************************************* 17 : 1986 ************************************************** 18 : 1919 ************************************************* 19 : 1647 ****************************************** 20 : 1526 *************************************** 21 : 1360 ********************************** 22 : 1231 ******************************* 23 : 1004 ************************* 24 : 802 ******************** 25 : 765 ******************* 26 : 597 *************** 27 : 486 ************ 28 : 334 ******** 29 : 253 ****** 30 : 5 ================================================================================