Changeset 150 for trunk/README

Show
Ignore:
Timestamp:
08/25/06 15:30:34 (7 years ago)
Author:
mk
Message:

Merging mk branch into the trunk.

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • trunk/README

    r5 r150  
    1212 
    1313 * whether the package can be downloaded from PyPI given its name 
    14  * whether the package can be downloaded from a full URL 
    1514 * whether the package can be unpacked 
    16  * whether the unpack directory is the same as the package name 
    1715 * whether the package can be installed into an alternate directory 
    1816 * existence of certain files such as README, INSTALL, LICENSE, setup.py etc. 
    19  * existence of certain directories such as doc, test, demo, examples 
    2017 * percentage of modules/functions/classes/methods with docstrings 
    21  * percentage of functions/methods that are unit tested (not currently  
    22    implemented) 
    23  * average pylint score for all non-test and non-demo modules 
     18 * pylint score 
     19 * ... and many others 
    2420 
    2521Currently, the Cheesecake index is computed for invidual packages obtained  
    …… 
    7470 
    7571If the package can be successfully downloaded and unpacked, a log file is 
    76 created in the sandbox directory and named <package>.log (e.g. the log file  
    77 for twill-0.7.4.tar.gz is /tmp/cheesecake_sandbox/twill-0.7.4.tar.gz.log). 
    78 The log file is not automatically deleted after the Cheesecake index is 
    79 computed, since its purpose is to be inspected for debug information. 
     72created in the system /tmp directory and named <package>.log (e.g. the log file  
     73for twill-0.7.4.tar.gz is /tmp/twill-0.7.4.tar.gz.log). 
     74The log file is automatically deleted after the Cheesecake index is 
     75computed, except for situations when errors have occured. 
    8076 
    8177Command-line examples: 
    …… 
    9894    For more options, run cheesecake.py with -h or --help. 
    9995 
     96Requirements 
     97------------ 
     98 
     99* `pylint <http://www.logilab.org/projects/pylint>`_ is required for 
     100  part of the code kwalitee index computation  
     101* `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_ is 
     102  required for the installability index computation 
     103 
    100104Obtaining the source code 
    101105------------------------- 
    102106 
    103 The Cheesecake project has not yet been released as a tarball or 
    104 a Python egg. You can obtain the source code from SourceForge via CVS:: 
    105  
    106   cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/cheesecake co -P cheesecake 
    107  
    108 Mailing list 
    109 ------------ 
    110  
    111 Developer mailing list: http://lists.sourceforge.net/lists/listinfo/cheesecake-devel 
     107You can get the source code via svn:: 
     108 
     109  svn co http://svn.pycheesecake.org/trunk cheesecake 
     110 
     111*Note*: make sure you indicate the target directory when you do the svn checkout, 
     112otherwise the cheesecake package files will be checked out directly in your 
     113current directory. 
     114 
     115You may want to modify your subversion client configuration to automatically 
     116expand tags, like $Id$, $Author$ etc. To do so add following two lines to your 
     117``/.subversion/config``:: 
     118 
     119  enable-auto-props = yes 
     120 
     121in [miscellany] section, and:: 
     122 
     123  *.py = svn:eol-style=native;svn:keywords=Author Date Id Revision 
     124 
     125in [auto-props] section. 
     126 
     127Documentation 
     128------------- 
     129 
     130The most recent code documentation should be always available 
     131at http://agilistas.org/cheesecake/mk/docs/. You can also generate 
     132this documentation directly from the Cheesecake sources. Run this command 
     133from the main source directory:: 
     134 
     135  sh support/generate_docs.sh . 
     136 
     137:Note: Generating documentation requires `epydoc <http://epydoc.sourceforge.net/>`_ 
     138       tool installed. 
     139 
     140Unit tests 
     141---------- 
     142 
     143We use `nose <http://somethingaboutorange.com/mrl/projects/nose/>`_ for automatic 
     144testing of our project, so if you want to test Cheesecake on your machine, please 
     145install that first. Running the standard set of Cheesecake unit test is as easy as:: 
     146 
     147  python setup.py test 
     148 
     149This command is equivalent to:: 
     150 
     151  nosetests --verbose --with-doctest --doctest-tests --include unit --exe 
     152 
     153We also have a set of functional tests, which can be run by issuing this command:: 
     154 
     155  nosetests --verbose --include functional 
     156 
     157Functional tests can take a bit longer to complete, as they test cheesecake_index 
     158script as a whole (as opposed to testing modules and classes separately). 
     159 
     160If you happen to find any of our tests failing, please don't hesitate to contact 
     161us, either via 
     162`cheesecake-devel mailing list <http://lists2.idyll.org/listinfo/cheesecake-dev>`_ 
     163or via `Cheesecake Trac <http://pycheesecake.org/>`_. 
     164 
     165Buildbot 
     166-------- 
     167 
     168A buildbot is happily running svn updates and unit tests. Check it out 
     169`here <http://agilistas.org:8888/>`_. 
     170 
     171Mailing lists 
     172------------- 
     173 
     174* Developer mailing list: http://lists2.idyll.org/listinfo/cheesecake-dev 
     175* User mailing list: http://lists2.idyll.org/listinfo/cheesecake-users 
    112176 
    113177License 
    …… 
    120184http://www.opensource.org/licenses/PythonSoftFoundation.php. 
    121185 
    122 Author contact info 
    123 ------------------- 
     186Authors contact info 
     187-------------------- 
    124188 
    125189Grig Gheorghiu 
    126190 
    127 Email: <grig at gheorghiu dot net> 
    128  
    129 Web site:  http://agiletesting.blogspot.com 
     191:Email: <grig at gheorghiu dot net> 
     192:Web site: http://agiletesting.blogspot.com 
     193 
     194Michal Kwiatkowski 
     195 
     196:Email: <ruby at joker.linuxstuff.pl> 
     197:Web site: http://joker.linuxstuff.pl 
     198 
     199Note: clipart for the cheesecake slice logo used with permission from 
     200Kazumi Hatasa, Director, the Japanese School at Middlebury College, 
     201Purdue University. 
    130202 
    131203Algorithm for computing the Cheesecake index 
    132204-------------------------------------------- 
    133205 
    134 The cheesecake.py module uses the following constants:: 
    135  
    136  INDEX_PYPI_DOWNLOAD = 50 
    137  INDEX_PYPI_DISTANCE = 5 
    138  INDEX_URL_DOWNLOAD  = 25 
    139  INDEX_UNPACK        = 25 
    140  INDEX_UNPACK_DIR    = 15 
    141  INDEX_INSTALL       = 50 
    142  INDEX_FILE_CRITICAL = 15 
    143  INDEX_FILE          = 10 
    144  INDEX_FILE_PYC      = 20 
    145  INDEX_DIR_CRITICAL  = 25 
    146  INDEX_DIR           = 20 
    147  INDEX_DIR_EMPTY     = 5 
    148  
    149  MAX_INDEX_DOCSTRINGS = 100 # max. percentage of modules/classes/methods/functions with docstrings 
    150  MAX_INDEX_PYLINT     = 100 # max. pylint score 
    151  
    152 **Step 0** 
    153  
    154 Initialize the Cheesecake index to 0. Also initialize to 0  
    155 the partial Cheesecake indexes for installability, documentation  
    156 and code kwalitee. 
    157  
    158 Compute the maximum overall Cheesecake index that can be reached by  
    159 any given package, which is the sum:: 
    160  
    161  INDEX_PYPI_DOWNLOAD +  
    162  INDEX_UNPACK + INDEX_UNPACK_DIR +  
    163  INDEX_INSTALL + 
    164  MAX_INDEX_DOCSTRINGS + MAX_INDEX_PYLINT +  
    165  (INDEX_FILE * number_of_expected_files) + 
    166  (INDEX_FILE_CRITICAL * number_of_expected_critical_files) + 
    167  (INDEX_DIR * number_of_expected_dirs) + 
    168  (INDEX_DIR_CRITICAL * number_of_expected_critical_dirs) 
    169  
    170 Compute the maximum Cheesecake index for installability, which is the sum:: 
    171  
    172  INDEX_PYPI_DOWNLOAD +  
    173  INDEX_UNPACK + INDEX_UNPACK_DIR +  
    174  INDEX_INSTALL 
    175  
    176 Compute the maximum Cheesecake index for documentation, which is the sum:: 
    177  
    178  (INDEX_FILE * number_of_expected_files) + 
    179  (INDEX_FILE_CRITICAL * number_of_expected_critical_files) + 
    180  (INDEX_DIR * number_of_expected_dirs) + 
    181  (INDEX_DIR_CRITICAL * number_of_expected_critical_dirs) + 
    182  MAX_INDEX_DOCSTRINGS 
    183  
    184 Compute the maximum Cheesecake index for code kwalitee, which is currently:: 
    185  
    186  MAX_INDEX_PYLINT 
    187  
    188 **Step 1a** 
    189  
    190 If short name of the package was specified with ``-n`` or ``--name``,  
    191 try to download the package from the PyPI index page by following the links to  
    192 the package home page and the package download URL (this is accomplished  
    193 using setuptools utilities). 
    194  
    195 If not successful, exit with a Cheesecake index of 0. If successful and  
    196 package was found at the Cheese Shop, add ``INDEX_PYPI_DOWNLOAD`` to  
    197 the overall Cheesecake index and to the installability Cheesecake index. 
    198  
    199 If successful but package was not found at the Cheese Shop, add  
    200 ``INDEX_PYPI_DOWNLOAD - (INDEX_PYPI_DISTANCE * number_of_links_to_package)`` 
    201 to the overall Cheesecake index and to the installability Cheesecake index. 
    202  
    203 **Step 1b** 
    204  
    205 If full URL of the package was specified with ``-u`` or ``--url``,  
    206 try to download the package from the specified URL. 
    207  
    208 If not successful, exit with a Cheesecake index of 0. If successful,  
    209 add ``INDEX_URL_DOWNLOAD`` to the overall Cheesecake index and to  
    210 the installability Cheesecake index. 
    211  
    212 **Step 1c** 
    213  
    214 If path to package on local file system was specified with ``-p`` or 
    215 ``--path``, copy the package to the sandbox directory. 
    216  
    217 **Step 2** 
    218  
    219 Unpack the package (currently supported archive types are zip and  
    220 tar.gz/tgz; in the near future we will support Python Eggs.) 
    221  
    222 If not successful, exit with a Cheesecake index of 0. If successful, add  
    223 ``INDEX_UNPACK`` to the overall Cheesecake index and to the installability  
    224 Cheesecake index. 
    225  
    226 **Step 3** 
    227  
    228 Check that the unpack directory has the same name as the package name 
    229 (i.e. when unpacking twill-0.7.4.tar.gz, we expect the unpack directory  
    230 to be twill-0.7.4.) 
    231  
    232 If the unpack directory name is the same as the package name, add  
    233 ``INDEX_UNPACK_DIR`` 
    234 to the overall Cheesecake index and to the installability Cheesecake index. 
    235  
    236 **Step 4** 
    237  
    238 Install the package to a temporary directory in a non-default location.  
    239 If successful, add ``INDEX_INSTALL`` to the overall Cheesecake index and to the 
    240 installability Cheesecake index. 
    241  
    242 **Step 5** 
    243  
    244 Check for existence of specific files.  
    245 For each file found, add ``INDEX_FILE`` to the overall  
    246 Cheesecake index and to the documentation Cheesecake index.  
    247 If the file is deemed critical, add ``INDEX_FILE_CRITICAL`` instead. 
    248  
    249 The following special files ("cheese_files") are currently checked:: 
    250  
    251     cheese_files = ["install", "changelog", 
    252                     "news", "faq", 
    253                     "todo", "thanks", "announce", 
    254                     "ez_setup.py", 
    255                    ] 
    256  
    257 The following files are currently deemed critical:: 
    258  
    259     critical_cheese_files = ["readme", "license", "setup.py"] 
    260  
    261 To check if a file FILE is among the cheese files, the following regular 
    262 expression is used:: 
    263  
    264     re.search(r"^%s(\.txt)*" % cheese_file, file, re.IGNORECASE) 
    265  
    266 **Step 6** 
    267  
    268 Check for existence of specific directories.  
    269 For each directory found, add ``INDEX_DIR`` to the overall Cheesecake  
    270 index and to the documentation Cheesecake index.  
    271 If the directory is deemed critical, add ``INDEX_DIR_CRITICAL`` instead. 
    272 If the directory is found empty, add ``INDEX_DIR_EMPTY`` instead. 
    273  
    274 The following directories ("cheese_dirs") are currently checked:: 
    275  
    276     cheese_dirs = ["example", "demo"] 
    277  
    278 The following directories are currently deemed critical:: 
    279  
    280     critical_cheese_dirs = ["doc", "test"] 
    281  
    282 To check if a directory DIR is among the cheese directories,  
    283 the following regular expression is used:: 
    284  
    285     re.search(r"^%s" % cheese_dir, DIR, re.ignorecase) 
    286  
    287 **Step 7** 
    288  
    289 Check for existence of .pyc files. If found, decrease the score  
    290 by subtracting ``INDEX_FILE_PYC`` from the overall Cheesecake index  
    291 and from the documentation Cheesecake index. 
    292  
    293 **Step 8** 
    294  
    295 Compute the percentage of modules/classes/methods/functions that have  
    296 docstrings associated with them. Only Python modules that are not in test,  
    297 doc, demo and example directories are checked.  
    298 Round up the percentage and add it to the overall Cheesecake index and to the 
    299 documentation Cheesecake index. 
    300  
    301 **Step 9** 
    302  
    303 If pylint is present on the system, run pylint against all Python files  
    304 that are not in the test, docs or demo directories.  
    305 Average the non-negative pylint scores, multiply the average by 10 and  
    306 add it to the overall Cheesecake index and to the code kwalitee  
    307 Cheesecake index. 
    308   
    309 **Step 10** 
    310  
    311 For each of the partial Cheesecake index types (installability,  
    312 documentation and code kwalitee), display the absolute Cheesecake  
    313 index for that type as the sum of all indexes of that type computed in  
    314 the previous steps.  
    315 Also display the relative Cheesecake index for that type as the percentage 
    316 of ``(absolute_index / maximum_index)``. 
    317  
    318 Display the absolute Cheesecake index for the package as the sum of all 
    319 indexes computed in the previous steps. Also display the relative Cheesecake 
    320 index for the package as the percentage of ``(absolute_index / maximum_index)``. 
     206The overall Cheesecake score is the sum of values of 3 main indexes 
     207(installability, documentation and code kwalitee). The values of these 
     208indexes rely on values of their subindexes and so on. The whole index tree 
     209and corresponding values for each leaf are presented below: 
     210 
     211* Installability 
     212 
     213  * package is listed on and can be downloaded from PyPI: 50 
     214  * package can be downloaded from given URL: 25 
     215  * package can be unpacked without problems: 25 
     216  * unpacked package directory is the same as package name: 15 
     217  * package has setup.py: 25 
     218  * package can be installed to given directory via "setup.py install": 50 
     219  * package contain generated files, like .pyc: -20 
     220 
     221* Documentation 
     222 
     223  * package contain files listed below 
     224 
     225    * README: 30 
     226    * LICENCE/COPYING: 30 [#oneof]_ 
     227    * ANNOUNCE/CHANGELOG: 20 [#oneof]_ 
     228    * INSTALL: 20 
     229    * AUTHORS: 10 
     230    * FAQ: 10 
     231    * NEWS: 10 
     232    * THANKS: 10 
     233    * TODO: 10 
     234 
     235  * package contain directories listed below 
     236 
     237    * doc/docs: 30 [#oneof]_ 
     238    * test/tests: 30 [#oneof]_ 
     239    * demo/example/examples: 10 [#oneof]_ 
     240 
     241  * code is documented by docstrings: 100 [#docstrings]_ 
     242  * docstrings have proper formatting (like epytext or reST): 30 [#formatted]_ 
     243 
     244* Code Kwalitee 
     245 
     246  * package has high pylint score: 50 
     247  * package has unit tests: 30 
     248 
     249The final score depends on how well the package scores for all indexes 
     250listed above. The score is presented in absolute range (number of points) 
     251and relative (percent of points obtained compared to maximum possible points). 
     252 
     253.. [#oneof] It is enough for a package to contain only one of listed files. 
     254.. [#docstrings] Number of points is proportional to percent of documentable objects 
     255   (module, class or function) that have docstrings. For example, if 
     256   you have 50 documentable objects and 32 of them have docstrings 
     257   your code will get 64 points (because 64% of objects are documented). 
     258.. [#formatted] Number of points depends on number of docstrings that are found 
     259   to contain one of known markup. Currently ReST, epytext and javadoc are 
     260   recognized. We give 10 points for 25% of formatted docstrings, 20 points 
     261   for 50% and 30 points for 75%. 
    321262 
    322263Sample output 
    …… 
    325266:: 
    326267 
    327  $ python cheesecake.py -n Durus 
    328  [cheesecake:console] Trying to download package durus from PyPI using setuptools utilities 
    329  [cheesecake:console] Downloaded package Durus-3.1.tar.gz from http://www.mems-exchange.org/software/durus/Durus-3.1.tar.gz 
    330  [cheesecake:console] Detailed info available in log file /tmp/cheesecake_sandbox/durus.log 
    331  [cheesecake:console] A given package can currently reach a MAXIMUM number of 555 points 
    332  [cheesecake:console] Starting computation of Cheesecake index for package 'Durus-3.1.tar.gz' 
    333  
    334  [cheesecake:console] Starting computation of INSTALLABILITY index (max. points = 140) 
    335  index_pypi_download .....................  45 (downloaded package Durus-3.1.tar.gz following 1 link from PyPI) 
    336  index_unpack ............................  25 (package untar-ed successfully) 
    337  index_unpack_dir ........................  15 (unpack directory is Durus-3.1 as expected) 
    338  index_install ...........................  50 (package installed in /tmp/cheesecake_sandbox/tmp_install_Durus-3.1) 
    339  --------------------------------------------- 
    340  INSTALLABILITY INDEX (ABSOLUTE) ......... 135 
    341  INSTALLABILITY INDEX (RELATIVE) .........  96 (135 out of a maximum of 140 points is 96%) 
    342  
    343  [cheesecake:console] Starting computation of DOCUMENTATION index (max. points = 415) 
    344  index_file_announce .....................   0 (file not found) 
    345  index_file_changelog ....................   0 (file not found) 
    346  index_file_ez_setup.py ..................   0 (file not found) 
    347  index_file_faq ..........................  10 (file found) 
    348  index_file_install ......................  10 (file found) 
    349  index_file_license ......................  15 (critical file found) 
    350  index_file_news .........................   0 (file not found) 
    351  index_file_readme .......................  15 (critical file found) 
    352  index_file_setup.py .....................  15 (critical file found) 
    353  index_file_thanks .......................   0 (file not found) 
    354  index_file_todo .........................   0 (file not found) 
    355  index_dir_demo ..........................   0 (directory not found) 
    356  index_dir_doc ...........................  25 (critical directory found) 
    357  index_dir_example .......................   0 (directory not found) 
    358  index_dir_test ..........................  25 (critical directory found) 
    359  index_docstrings ........................  42 (found 104/249=41.77% modules/classes/methods/functions with docstrings) 
    360  --------------------------------------------- 
    361  DOCUMENTATION INDEX (ABSOLUTE) .......... 157 
    362  DOCUMENTATION INDEX (RELATIVE) ..........  37 (157 out of a maximum of 415 points is 37%) 
    363   
    364  [cheesecake:console] Starting computation of CODE KWALITEE index (max. points = 100) 
    365  index_pylint ............................  64 (average score is 6.30 out of 10) 
    366  --------------------------------------------- 
    367  CODE KWALITEE INDEX (ABSOLUTE) ..........  64 
    368  CODE KWALITEE INDEX (RELATIVE) ..........  64 (64 out of a maximum of 100 points is 64%) 
    369   
    370  ============================================= 
    371  OVERALL CHEESECAKE INDEX (ABSOLUTE) ..... 356 
    372  OVERALL CHEESECAKE INDEX (RELATIVE) .....  64 (356 out of a maximum of 555 points is 64%) 
    373  
     268    $ cheesecake_index -n Durus 
     269    py_pi_download .........................  50  (downloaded package Durus-3.4.1.tar.gz following 1 link from http://www.mems-exchange.org/software/durus/Durus-3.4.1.tar.gz) 
     270    unpack .................................  25  (package unpacked successfully) 
     271    unpack_dir .............................  15  (unpack directory is Durus-3.4.1 as expected) 
     272    setup.py ...............................  25  (setup.py found) 
     273    install ................................  50  (package installed in /tmp/cheesecakeUrZH1A/tmp_install_Durus-3.4.1) 
     274    generated_files ........................   0  (0 .pyc and 0 .pyo files found) 
     275    --------------------------------------------- 
     276    INSTALLABILITY INDEX (ABSOLUTE) ........ 165 
     277    INSTALLABILITY INDEX (RELATIVE) ........ 100  (165 out of a maximum of 165 points is 100%) 
     278 
     279    required_files ......................... 170  (5 files and 2 required directories found) 
     280    docstrings .............................  33  (found 121/369=32.79% objects with docstrings) 
     281    formatted_docstrings ...................   0  (found 6/369=1.63% objects with formatted docstrings) 
     282    --------------------------------------------- 
     283    DOCUMENTATION INDEX (ABSOLUTE) ......... 203 
     284    DOCUMENTATION INDEX (RELATIVE) .........  58  (203 out of a maximum of 350 points is 58%) 
     285 
     286    pylint .................................  33  (pylint score was 6.59 out of 10) 
     287    unit_tested ............................  30  (have unit tests) 
     288    --------------------------------------------- 
     289    CODE KWALITEE INDEX (ABSOLUTE) .........  63 
     290    CODE KWALITEE INDEX (RELATIVE) .........  79  (63 out of a maximum of 80 points is 79%) 
     291 
     292 
     293    ============================================= 
     294    OVERALL CHEESECAKE INDEX (ABSOLUTE) .... 431 
     295    OVERALL CHEESECAKE INDEX (RELATIVE) ....  72  (431 out of a maximum of 595 points is 72%) 
     296 
     297Case study: Cleaning up PyBlosxom 
     298--------------------------------- 
     299 
     300Many thanks to Will Guaraldi for writing 
     301`this article <http://pycheesecake.org/wiki/CleaningUpPyBlosxom>`_ about his 
     302experiences in using Cheesecake to clean up and improve the structure of his 
     303PyBlosxom package. 
     304     
    374305Future plans 
    375306------------ 
    …… 
    377308index measurement, followed by other metrics inspired from the  
    378309`kwalitee indicators <http://cpants.dev.zsi.at/kwalitee.html>`_.  
    379 Please edit the `IndexMeasurementIdeas <http://tracos.org/cheesecake/wiki/IndexMeasurementIdeas>`_ 
     310Please edit the `IndexMeasurementIdeas <http://pycheesecake.org/wiki/IndexMeasurementIdeas>`_ 
    380311Wiki page to add things that you would like to see covered  
    381312by the Cheesecake metrics. 
    382313 
    383 .. footer:: Generated with rst2html.py from the  
    384    `docutils <http://docutils.sourceforge.net/>`_  
    385    distribution. Last modified 2005-12-20 by  
    386    `Grig Gheorghiu <http://agiletesting.blogspot.com>`_. 
     314.. footer:: Last modified 2006-08-21 by `Michal Kwiatkowski <http://joker.linuxstuff.pl>`_.