Bric::Hacker - A guide for Bricolage hackers.
$Revision: 1.37.2.9 $
$Date: 2004/02/06 20:00:14 $
This document is designed to provide information useful to Bricolage developers. If you've got questions about hacking Bricolage that aren't answered here please post to the Bricolage developer's mailing-list (see below) and tell us about it.
Bricolage has a number of mailing-lists that are relevant to developers:
This is the mailing-list for normal Bricolage users. This is a good place to go with questions about how the application works and should work. To subscribe go to:
http://lists.sourceforge.net/lists/listinfo/bricolage-general
This is the place to discuss Bricolage development, propose changes, post patches and penetrate markets. To subscribe go to:
http://lists.sourceforge.net/lists/listinfo/bricolage-devel
This list gets an email for every commit to the CVS tree (see below for more about CVS). DO NOT POST TO THIS LIST! To subscribe:
http://lists.sourceforge.net/lists/listinfo/bricolage-commits
This list gets an email for every bug-report submitted to Bugzilla (see below for Bugzilla details). DO NOT POST TO THIS LIST! To subscribe:
http://lists.sourceforge.net/lists/listinfo/bricolage-bugs
You can often find folks hanging out and occasionally discussing development issues on the #bricolage channel on the Rhizomatic IRC network.
If you're developing Bricolage then you should be working with the latest code from CVS. You can browse the CVS tree at:
http://sourceforge.net/cvs/?group_id=34789You'll also find information there on connecting to the repository to checkout a working copy.
Bricolage has a Bugzilla server dedicated to it:
http://bugzilla.bricolage.cc/You should use this system to report bugs in Bricolage. If you're looking for something to do you can also use the system to find open bugs and fix them. For more things to do see Bric::ToDo.
Patches should be generated using a cvs diff -u command on each of the files modified, from the root directory. For example, if you made changes to lib/Bric/Changes.pod and comp/foo.mc, you would generate a diff by running this command from the root of your CVS checkout:
cvs diff -u lib/Bric/Changes.pod comp/foo.mc > patch.txtIf you created one or more new files in your changes then you'll have to add them to the patch separately using normal diff against /dev/null. For example, if you created the file inst/upgrade/1.9.1/solve_fermat.pl then you would add this to patch.txt with:
diff -u /dev/null inst/upgrade/1.9.1/solve_fermat.pl >> patch.txtAlways create patches using the most recent CVS version if possible. Send your patches to the bricolage-devel list mentioned above.
Patches created using the method above can be applied using patch with the -p0 option from the root of your CVS checkout:
patch -p0 < patch.txtMake sure you check the results with cvs diff before committing.
Try to follow the style of the existing Bricolage code. Except where it is bad, of course. Basically, write in the style you see in most Perl books and documentation, particularly perlstyle.
Although historically the Bricolage code has not enforced whitespace rules, we now request that you use 4-space indents (2 spaces for continued lines) and discourage the use of tabs. The following settings for some of the more popular editors are thus recommended while editing Bricolage source code.
We strongly recommend that you use cperl-mode while editing Bricolage sources in Emacs. Grab the latest version from the CPAN, install it, and then place the following in your ~/.emacs file:
(custom-set-variables
'(case-fold-search t)
'(cperl-indent-level 4)
'(cperl-continued-statement-offset 2)
'(cperl-tab-always-indent t)
'(indent-tabs-mode nil))Also, if you'd like to take advantage of the full functionality of cperl-mode and have it automatically parse all Perl source files, add these settings, as well:
(defalias 'perl-mode 'cperl-mode)
(setq auto-mode-alist
(append
'(("\\.\\([pP]\\([Llm]\\|erl\\)\\|al\\|pod\\)\\'" . cperl-mode))
auto-mode-alist))
(setq cperl-hairy t)
(setq interpreter-mode-alist (append interpreter-mode-alist
'(("miniperl" . cperl-mode))))When editing Bricolage Mason components, mmm-mode can help. It's Mason mode will parse Mason component files and use sgml-mode in HTML spaces and cperl-mode in Mason blocks. Grab it from http://mmm-mode.sourceforge.net/, install it, and then add the following to your ~/.emacs file to have it automatically parse your Bricolage Mason component files:
(add-to-list 'load-path "/usr/local/share/emacs/site-lisp")
(require 'mmm-mode)
(require 'mmm-mason)
(setq mmm-global-mode 'maybe)
(add-to-list 'auto-mode-alist '("/usr/local/bricolage/comp" . sgml-mode))
(mmm-add-mode-ext-class 'sgml-mode "/usr/local/bricolage/comp" 'mason)And if you need to examine the Mason object files created by Bricolage in order to chase down bugs and such, you can use the cperl-mode in those files by adding this to your ~/.emacs file:
(add-to-list 'auto-mode-alist '("/usr/local/bricolage/data/obj" . cperl-mode))Rafael Garcia-Suarez has written a Vim indent macro which (for the most part) duplicates the behavior of Emacs perl-mode. It is now, as of Vim 6.0, included in the Vim distribution and should be found in $VIMRUNTIME/indent/perl. The easiest way to use it though is to place the following line in your .vimrc:
source $VIMRUNTIME/indent.vimYou'll also need to add these lines.
set tabstop=8
set softtabstop=4
set shiftwidth=4
set expandtabThe first three lines make Vim duplicate the behavior of Emacs in creating the appearance of 4 space tabs with a mix of tabs and spaces. This is necessary for reading older Bricolage files which were written this way, and haven't yet been re-tabbed.
The expandtab setting does the Right Thing under the new rules, in that it doesn't use tabs at all, only spaces.
Keep subroutines short. Each subroutine should handle one task. For example, if you have a subroutine get_foo, and getting "foo" requires getting "bar" and "qux", then make two more subroutines get_bar and get_qux and put them in the "private functions" section. Also if you duplicate some code in several subroutines, factor it out into another subroutine. It's easier to maintain one subroutine than to maintain three.
Limit the width of your code to 80 characters if possible. This makes them easier to read. Keeping subroutines short helps to meet this goal, as well.
Make enough comments so that someone maintaining your code can understand what is going on. Comments shouldn't be redundant with the code. They should explain non-obvious code. Comments can be bad in some cases if code changes and the corresponding comment is not kept in sync. So keep the comments in sync!
Use Perl idioms if it is clear and concise, but use them with care. Implicit variables can be slick, but hard to understand; add a comment if you use uncommon ones.
Use Mason components only for display, and put business logic into library modules and callback components. But only for as long as callbacks are in components! We hope to move them into libraries soon, at which time this rule will become even more rigid.
For POD, generally you copy/paste it from another module. Recently we are trying to cut out some of useless things like BE<lt>Side Effects:E<gt> NONE. For a canonical example, see Bric::Biz::Site.
Avoid magic values, i.e. don't hard-code numbers into code. It tends to become a maintenance problem if you put the number 1023 throughout a module and then somewhere else you have 1021; is 1021 related to 1023 (1023 - 2), or is it just another random number? And what does 1023 mean? Put a constant at the top of the module and use that, instead.
In the end, just try to follow the existing code style, and take into consideration any feedback you get from the mailing list when patches are submitted.
All Bricolage patches should be accompanied by the necessary tests. You are strongly encouraged to write comprehensive tests that thoroughly test whatever changes or additions you make to the Bricolage API. You are also strongly encouraged to write tests for the current API, if you find that adequate tests have not yet been written (and this is true in a great many places, unfortunately). Although Bricolage does not yet support UI tests, we take our API testing seriously, and so should you. Here's what you need to know to write tests for Bricolage.
Bricolage contains a test suite based on Test::Class. The test classes live in the t/Bric directory, and all subclass Bric::Test::Base. If you're familiar with Test::More, then the syntax for how the Test::Class-based classes work should be pretty readily apparent. See the Test::Class documentation for details. It's definitely worth a read. If you haven't used Test::More, its documentation is also a must-read.
Bricolage has two different sets of tests, those run by make test and those run by make devtest. The former are stand-alone tests that don't rely on the presence of a Bricolage database to be run. They can thus be run before make install. The idea here is that we offer a basic set of tests that can be run via the standard Perl installation pattern of
perl Makefile.PL
make
make test
make installThe tests run by <make devtest> are intended to be run during development. They require that a Bricolage database be installed and running. They will SELECT, INSERT, UPDATE, and DELETE data from that database, so make devtest should never be run against a production database. Indeed, you should in general have a clean, fresh database installation to run the tests against. Although the test suite makes every effort to clean up after itself by DELETEing data it has added to the database, it is unfortunately imperfect, and extra data will be left, especially in Group-related and Attribute-related tables.
So, once more, make devtest should never be run against a production database.>. 'Nuff said.
All of the tests are run by the inst/runtests.pl script, which in turn tells Test::Harness to execute t/Bric/Test/Runner.pm. This file will find all the necessary test classes, load them, and then run their tests. inst/runtests.pl takes a number of arguments to simplify the running of scripts, including a list of test files or classes to run, so that you can just run the tests you need to run while you're developing new tests. perldoc inst/runtests.pl for more information.
Be sure to use the testing base classes in your test classes. For non-database dependent tests (which are always in files named Test.pm, use Bric::Test::Base. For development tests, use Bric::Test::DevBase, which inherits from Bric::Test::Base. Be sure to read the documentation in these classes, as it will help you to write your tests more effectively. Pay special attention to the methods added to Bric::Test::DevBase, as they're there to help you clean up any new records you've added to the database.
And finally, when you do provide patches to Bricolage, along with the new tests to test them, make sure that all existing tests pass, as well. This means that you should always run make devtest on a fresh database build with your changes. Furthermore, if your patch involves changes to the database, you should provide the necessary upgrade script in inst/upgrade/, and also run the tests against a database that has been built from Bricolage sources untouched by your patch, and then upgraded by your upgrade script. This will help to ensure that your upgrade script modifies the database in the same way as your patch modifies the Bricolage SQL files.
And with that said, happy testing!
Bricolage is a complex application and debugging can be difficult. Here are some tips to help you find bugs faster:
Turn on QA_MODE in your bricolage.conf file. This will cause the UI to display a bunch of useful data at the bottom of every page, including session state, cache state, cookies, and GET and POST args. QA_MODE turns on PerlWarn (use warnings) and causes error messages to include more information. QA_MODE also enables Apache::Status at the URL /perl-status which allows you can to examine the state of the Perl interpreter inside Apache.
Set the NO_TOOLBAR directive to "0" or "Off" in bricolage.conf to keep Bricolage from popping up a new browser window without a toolbar.
Run Bricolage under the Perl debugger using the debug command with bric_apachectl:
bric_apachectl debugThis will start Apache in single-process mode (httpd -X) and setup Apache::DB to start the debugger on the each hit to the server. You'll need to install the Apache::DB Perl module to use this command.
To run Bricolage under DDD (http://www.gnu.org/software/ddd/), start ddd as root and load bin/bric_apachectl. Give it the argument "debug" and run it. When you issue a hit to the server the debugger will stop on the first line of Bric::App::Handler::handler(). From there you can set breakpoints inside Bricolage and debug normally.
If you prefer to run without the debugger using only the Apache single-process mode, then run Bricolage using the command
bric_apachectl singleSet DBI_DEBUG and DBI_CALL_TRACE to 1 in your bricolage.conf file. DBI_DEBUG records every database call in the logs complete with SQL and arguments. DBI_CALL_TRACE adds a a subroutine call trace for each statement showing where the database call originated. This generates a lot of data but it can be very helpful.
Look at the database directly using psql. Many bugs in Bricolage can only be successfully diagnosed by examining records created in the database.
Bricolage has two separate profiling systems that you can use to extract performance data:
To run Bricolage under the Devel::Profiler module set the PROFILE variable on in your bricolage.conf file and restart your server. This will create a profiler/$$/tmon.out file in your Apache log directory for each server process. You can use dprofpp to analyze these files after the server has been stopped with bric_apachectl stop. See the Devel::Profiler documentation for more details.
The database profiler is activated by enabling the DBI_PROFILE option in bricolage.conf. This causes database profiling traces to be written to the Apache error log during requests. You can then use bric_dbprof to analyze these trace. See bric_dbprof for details.
CAUTION: Neither of these options is appropriate for a production system.
A relatively simple way to contribute to Bricolage is to provide actions and movers. These are plugin modules that can add new functionality to Bricolage without needing to make changes to the existing API.
An "action" is an act that is performed on files before they are distributed. Say you want to clean the HTML of all of your HTML files before they're distributed. You'll need to create an action to do this. Consult Bric::Dist::Action for information on how to create actions.
Say you need to distribute files via a protocol that Bricolage doesn't currently support -- say, an new variant of FTP called "FooTP." You'll need to create a new mover. Consult Bric::Dist::Action::Mover for details on how to do that.
If you're a Bricolage developer with permission to commit to the CVS repository, you may occasionally have to merge changes from a release branch (where bug fixes are generally committed) into the trunk. Here's how to do it with a minimum of hassle.
Determine if the branch checkout has been merged before. To do so, execute this cvs command in the branch checkout:
cd bricolage-branch-checkout/
cvs -z3 -q status -v READMEThis command will output the status of the README file, something like this:
===================================================================
File: README Status: Up-to-date
Working revision: 1.16.2.1
Repository revision: 1.16.2.1 /cvsroot/bricolage/bricolage/README,v
Sticky Tag: rev_1_4 (branch: 1.16.2)
Sticky Date: (none)
Sticky Options: (none)
Existing Tags:
rev_1_4 (branch: 1.16.2)
rev_1_4_merge-2002-08-31 (revision: 1.16.2.1)
dev_1_3_3 (revision: 1.10)
rev_1_2_merge-2002-04-03 (revision: 1.2.2.4)
rel_1_2_3 (revision: 1.2.2.4)From this output, you can see that, for this branch (rev_1_4), the file was tagged with the merge tag "rev_1_4_merge-2002-08-31". Thus we can tell that the branch was last merged on August 31, 2002. If there had not been a tag with the same version number as the branch, then we could determine that it hadn't been merged before.
If the branch has not been merged into the trunk before, cd into a checkout from HEAD, make sure it's fully up-to-date, and do a simple merge:
cd bricolage-head-checkout/
cvs -z3 -q update
cvs -z3 -q update -d -kk -j rev_1_4Otherwise, do a merge from the most recently-dated merge tag. In the above example, that would be "rev_1_4_merge-2002-08-31".
cd bricolage-head-checkout/
cvs -z3 -q update
cvs -z3 -q update -d -kk -j rev_1_4_merge-2002-08-31 -j rev_1_4The -d option will allow new directories and their contents added to the branch to be merged into HEAD. Note also the use of the -kk option. This option tells CVS not to evaluate keywords in the files. This is crucial for merges, since keywords such as $Revision $ will be different. Not to worry, though; the keyword values are always restored before releasing a new version. See Creating Distributions below.
Resolve any conflicts (hopefully none, but they do happen occasionally) and then rebuild the database and run all of the tests:
make devtestIf there are any test failures, you'll need to fix those, too.
And finally, commit the changes to the trunk.
cvs -z3 -q commitTag the branch with today's date, so that future merges can decide to merge only from this date on.
cd bricolage-branch-checkout/
cvs -z3 -q tag rev_1_4_merge-CCYY-MM-DDBy following this methodology we should be able to minimize the number of conflicts we get between merges. See the Karl Fogel and Moshe Bar's book "Open Source Development with CVS, 2nd Edition", chapter 2, for a more detailed explanation of the whys and wheres of this approach to merging CVS branches.
If you are a Bricolage release manager and you're getting ready to release a new version, here are the steps you'll need to take to create a distribution tarball. First, add a date for the new release to Bric::Changes and commit it. Something like this:
=head1 VERSION 1.6.9 (2004-02-06)Next, tag the release in CVS:
cvs -z3 tag rel_1_x_xIf this is a major release, you'll need to create a new branch, so that it can be maintained for bug fixes separately, and then tag the release in that branch.
cvs -z3 tag -b rev_1_x
cvs -z3 tag rel_1_x_xNext, export the sources into a new directory. Be sure to use the -kkv option with export so that the version numbers are all properly populated.
cd /tmp
cvs -z3 export -r rel_1_x_x -kkv bricolage
cd bricolage
make distThis will create a distribution tarball in the bricolage directory. Copy this tarball somewhere, and do a full test with it, to make sure that Bricolage does indeed build and properly install itself.
cp bricolage-1.x.x.tar.gz /tmp/src
cd /tmp/src
tar zxvf bricolage-1.x.x.tar.gz
cd bricolage-1.x.x
perl Makefile.PL
make
# Shut down PostgreSQL.
make test
# Start PostgreSQL.
sudo make install
make devtest
sudo bric_apachectl start
# Log in to the UI and test it a bit.
sudo make uninstallBe sure to run make devtest after everything is installed to make sure that it is all functioning correctly. Users won't be running these tests, as they muck up the database. But that's okay for our validation of the release tarball. Use make uninstall to clean up the mess. If you have to go back and make any changes, be sure to update the tag of any files you change and commit to CVS, then do the whole thing over again. Once everything appears to be working properly, release!
Sam Tregar <stregar@about-inc.com>
David Wheeler <david@wheeler.net>