../../../tools/bin/corpus_analyze.prl run by knight on Mon Aug 2 15:11:58 1999 Analysis of experiments/10k/10k-train.snt: ================================================================================ CORPUS DATA: experiments/10k/10k-train.snt contains 9800 sentences (9792 unique). -------------------------------------------------------------------------------- TOKENS TYPES (C > 1)* -------------------------------------------------------------------------------- SOURCE: experiments/10k/10k-train.snt 152639 16010 (7852) TARGET: experiments/10k/10k-train.snt 135150 29513 (10272) -------------------------------------------------------------------------------- *The number of types which appear more than once. ================================================================================ CO-OCCURENCE DATA: -------------------------------------------------------------------------------- TOTAL UNIQUE (C > 1)* -------------------------------------------------------------------------------- experiments/10k/10k-train.snt 2443222 1219917 (230162) -------------------------------------------------------------------------------- *The number of co-occurences which appear more than once. ================================================================================ SENTENCE LENGTH DATA: -------------------------------------------------------------------------------- experiments/10k/10k-train.snt SOURCE: AVERAGE SENTENCE LENGTH: 15.5754081632653 LENGTH COUNT (* = 9 SENTENCES) 1 : 1 2 : 61 ****** 3 : 137 *************** 4 : 162 ****************** 5 : 214 *********************** 6 : 331 ************************************ 7 : 335 ************************************* 8 : 331 ************************************ 9 : 418 ********************************************** 10 : 467 *************************************************** 11 : 511 ******************************************************** 12 : 485 ***************************************************** 13 : 508 ******************************************************** 14 : 529 ********************************************************** 15 : 497 ******************************************************* 16 : 510 ******************************************************** 17 : 494 ****************************************************** 18 : 488 ****************************************************** 19 : 426 *********************************************** 20 : 379 ****************************************** 21 : 405 ********************************************* 22 : 361 **************************************** 23 : 327 ************************************ 24 : 289 ******************************** 25 : 271 ****************************** 26 : 259 **************************** 27 : 228 ************************* 28 : 187 ******************** 29 : 181 ******************** TARGET: AVERAGE SENTENCE LENGTH: 13.7908163265306 LENGTH COUNT (* = 9 SENTENCES) 1 : 5 2 : 80 ******** 3 : 168 ****************** 4 : 242 ************************** 5 : 308 ********************************** 6 : 418 ********************************************** 7 : 450 ************************************************** 8 : 523 ********************************************************** 9 : 560 ************************************************************** 10 : 534 *********************************************************** 11 : 574 *************************************************************** 12 : 587 ***************************************************************** 13 : 583 **************************************************************** 14 : 528 ********************************************************** 15 : 534 *********************************************************** 16 : 504 ******************************************************** 17 : 423 *********************************************** 18 : 429 *********************************************** 19 : 395 ******************************************* 20 : 370 ***************************************** 21 : 322 *********************************** 22 : 251 *************************** 23 : 240 ************************** 24 : 192 ********************* 25 : 187 ******************** 26 : 138 *************** 27 : 119 ************* 28 : 71 ******* 29 : 56 ****** 30 : 1 ================================================================================