May 03, 2008

DocProject Roadmap Part II

In my first DocProject Roadmap post I described my preliminary plans for DocProject 1.x and later versions.  In this post I'm going to discuss the project's current status and some updates to my plans for future versions.

1.11.0 RC

DocProject 1.11.0 RC will be deployed within a week after the upcoming Sandcastle release.  It contains several new features and bug fixes, and also a few breaking changes that I'll be sure to mention on the release page.

I expect a few changes to Sandcastle that may prompt the addition of a few new features to DocProject before the next release (e.g., support for Help 1.x and Help 2.x project management and, possibly, the ability to include raw HTML topics using Topic Explorer), but since I can't start working on them until Sandcastle is released, there's obviously going to be a delay.

A week should be more than enough time though since I've taken some time off from paid employment to work on my open source projects - a sacrifice that I can barely afford, but working on my own projects is more fun than writing line of business apps and I just couldn't resist after losing my only client anyway (they lost their biggest client first and had to suspend my services, maybe indefinitely).

Universal Installer

The next version of DocProject, 1.11.0 RC, can install DocProject for Visual Studio 2005 and 2008 simultaneously, including all supported Express editions.  The code base for DocProject 1.x and DocProject 2008 are now identical and as a result DocProject 2008 now uses the 1.x assemblies.

Ending Phase I

I've done a lot of refactoring in 1.11.0 RC, but I still plan to deploy it as the final release candidate (RC) for DocProject 1.x, with compiler optimizations enabled and without any debug symbol databases (.pdb).

Feedback is a must in this release so that I can iron out showstoppers and maybe even add some mission-critical features that people have been waiting for.  If you're a fan of DocProject and you use VS 2005 then now would be the time to get in your suggestions for DocProject 1.x since most of my focus in the next phase of development is going to be on Visual Studio 2008.

Unfortunately I didn't get to add some of the features to 1.11.0 RC as I had planned, but they may be added to 1.12.0 or, more likely, DocProject 2008 Beta 3.

First Production Release

I still expect 1.12.0 to be the first production release of DocProject, but now it will target both Visual Studio 2005 and 2008.  To support this I've reorganized source control, once again, and included the source for other projects as well, such as DocToMaml.  I've also decided not to package the source code with the installer anymore, starting with 1.11.0 RC.

My plans for DocProject 1.x after the first production release are to provide subsequent releases for bug fixes and small feature updates only, targeting both VS 2005 and 2008.  The major features that I have been planning will make their first appearance in DocProject 2008 Beta 3.

DocProject 2008

After I deploy 1.11.0 RC, development will continue on both 1.12.0 RTW and DocProject 2008 Beta 3 simultaneously, although the most notable of new features will only be included in DocProject 2008 Beta 3.  Some of these features are described below.

Please let me know which features you're most interested in so that I can prioritize.

Visual Studio Package

DocProject currently provides an Add-In that only uses VS automation, but I'd like to provide more integration to create a better Topic Explorer, Topic Editor and Topic Designer, and even a preliminary MAML editor.  I've done some initial research and have already come up with a plan to start development.

This is an exciting task to work on but I don't plan on the first release being extremely stable as this will be my first VS package; although, I do plan on reusing a fair amount of the automation code since it has already been tested.  I'll also incorporate much of DocProject's current architecture, including its high level of extensibility and its "openness".

Business logic, infrastructure and utilities will probably end up being mostly the same, but I do want to take this as an opportunity to update some of the designs that DocProject currently uses, like change buffering.

Standalone UI

The current DocProject External UI program relies on DocProjects and DocSites that have been created in Visual Studio.  For help authors that just-so-happen to be developers, this isn't a problem.  But for anyone that isn't a developer, the free Visual Studio Express edition works well but doesn't make much sense since it's obviously not designed just for building documentation, and certainly not for authoring it.

DocProject 2008 Beta 3 will use the new Visual Studio 2008 Shell to provide a tool that has all of DocProject's features, but without requiring Visual Studio to be installed.  I haven't decided yet whether I'm going to use integrated or isolated mode, but I'm leaning towards integrated.

This should be quite easy to create since DocProject is becoming a VS package anyway.  A blocking issue though is whether MSBuild is provided by the Shell redistributable.  If not, then I may just have to continue to deploy the External UI instead.  I'm also not sure whether the Shell provides C# and VB.NET source code editors, but if not it's no big deal since they aren't available in the current External UI either.  (The code editors would be used to edit the project's build process component.  I think this would be a great feature to have in the standalone app, if possible, but it's not a requirement.)

My ultimate goal for the standalone UI is to create a program that works with or without Visual Studio.  It must provide the ability to create new DocProjects and DocSites from existing MSBuild project templates, so they work in Visual Studio as well.  It will also provide all of DocProject's existing and upcoming features, including support for UI extensibility; e.g., providers can register new tool bars and tool windows.

Continuous Integration

A project option will be available to enable support for continuous integration.  When enabled, DocProject will copy all dependencies into the project, including DocProject and Sandcastle assemblies, configuration files, and the HTML Help Workshop program, allowing them to be easily committed to source control.  After a DocProject or DocSite is checked-in, another team member can get the latest version to their computer or to a build server and all the dependencies will be updated as well, without requiring any installation.

The only blocking issue of which I'm aware seems to be that Help 2.x requires installation.  DocProject does not use any of the command-line tools that are installed by the VS SDK, however, it does use the MS Help COM interfaces.  Unfortunately, to get the libraries and COM objects installed you must install the VS SDK, but to install the SDK you must first install Visual Studio.  I'll continue to research this when the time comes to see if I can find a workaround, but so far it doesn't look good.

Continuous Builds

I mentioned this in my previous roadmap post.  The idea is to detect only the changes made to APIs and XML documentation and have a web server that continuously rebuilds a live DocSite so that the updates are visible immediately to all team members.

This feature will require dependency detection and differencing algorithms for both API references and XML documentation.  One question is whether these algorithms will perform better than building entire documentation sets from scratch, but I believe that they certainly will in the scenarios where it really matters.

The goal is that for projects with many large assemblies there should be little delay to see the updates in documentation, at least compared to the very long delay that ensues when rebuilding entire documentation sets from scratch, which is currently the only option with Sandcastle.

To use this feature you would start be creating a new DocSite project and probably selecting some option in the New Project Wizard to indicate that you want continuous build support.  You would also select a VS solution file or an MSBuild project that the DocSite will use to automatically build sources.  In the last wizard step you would add all of the sources to the project as project references, like normal.  When finished configuring the project as usual, you would deploy it to an intranet web server that all team members can access over HTTP.

The live DocSite would trigger a rebuild using MSBuild at a configurable time interval.  This way MSBuild will detect updates automatically and the DocSite will always have the latest versions of all sources.  The DocSite will monitor changes to the sources and continuously rebuild only those topics that were affected.  It will also update the DocSiteContents.xml and DocSiteIndex.xml files, as well as the full-text search index.

One problem might be getting the latest source code onto the server so that the DocSite can rebuild it.  In a team scenario it's probably safe to assume that some sort of source control system is in use.  I'm not sure if it's possible, but if source control systems provide an event model that could be used to automatically get the latest version to the web server after each check-in, that would solve the problem.  Otherwise, manually getting the latest version to the server may be required.  It may be possible to access the source code over a network share (to a build server perhaps) so that it doesn't have to be stored on the web server, although I'm not sure if MSBuild works with UNC.

Using external sources instead of project references may be another option, but replacing the assemblies on a network share automatically might be just as difficult.  It could be done manually or, as with the solution for project references, using source control events to run a build after each (labeled?) check-in.

Localization

I've already put a few hours into researching localization support in DocProject.  What I discovered, and I'm not surprised, is that if I want an implementation that isn't a hack I'll have to add some new features to support dynamic changes to various configuration files during builds.  Anticipating this problem, I had decided to defer localization support into DocProject 2008.

I suspect that these new support features will be useful on their own as well, much in the same way that extension methods in C# 3.0 were probably added to support LINQ yet they are even useful by themselves.  The new features will add to DocProject's already high level of extensibility and will be customizable via the DocProject configuration file.

My goal for localization support is for a single DocProject or DocSite to produce multiple compiled help files for different locales in a single build.  This must be accomplished without affecting the current feature-set of DocProject, including the Sandcastle Build Component drop-down properties and the fact that you can currently change almost anything that you want in the BuildAssembler configuration files.  So in other words, the localization feature must be completely transparent when it's not being used.

All Sandcastle content items and user-written XML documentation comments will be localizable.  Once implemented, I plan to ask the community to create language packs for content items in various locales (Unfortunately, I only know English, and not so well ;).  A language pack will be a .zip file that contains one or more Sandcastle content documents and an XML manifest file, which describes the contents and rules for locale inheritance.  DocProject 2008 will be able to locate the packs in its installation directory, by default, or a user-configured location.

With language packs, DocProject can offer out-of-the-box localization for multiple languages so that help authors can simply check a box and build their project to produce multiple compiled help files, one per language.  Localized XML documentation comments can be added to the project's Help\Comments\[locale] folder (e.g., Help\Comments\en-GB) and it will be merged automatically with XML documentation comments from the source code (how it's merged will depend upon the Merge XML documentation project option, in the same exact way that it works now in the current version of DocProject).  Likewise, localized conceptual topics (MAML) may be added to the project's Help\Topics\[locale] folder.

Instead of trying to write down the many concepts related to implementing localization support that I have echoing around between my ears, I'll just post my notes in their raw form.  If you want more details, please let me know.

1. add content item mappings to DocProject config file:

  <item name="..." presentationNames="Visual Studio 2005, Sandcastle Hana" source="options-property-name"/>

  - Items that are added here are automatically removed from the content item documents when a project is

first created.
  - Before each build a dynamic content item document is generated in the working directory for all of the

items that are configured here.
  - Note that a configChange must be added to register the dynamic content item document.

2. Add a wizard step, before the Create Shared Content step, that allows a user to specify the values for

each of the pre-defined_enum_names:

  - Documentation set name
  - Running header text
  - Copyright
  - Company name
  - Default locale (combox box)

3. Add a Localization Management dialog.

  - Choose the different locales in which distinct documentation sets will be generated by selecting from a

drop down list, which will add the selection to another list.
    - one locale may be selected as the "working" locale for the dialog; UI elements will apply only to the

"working" locale.
    - a tree view, like solution explorer, shows only the folders that may be localized within the project;

e.g., Help\Topics, Help\Comments, etc.
    - one node may be selected at a time.
    - for the selected locale/node, a button can be clicked that will prompt the user to choose a locale from

which to import files, with the option of specifying whether the import should be performed recursively.
  - Choose a default locale (corresponds to the wizard step option indicated above).
  - A single build will produce all of the documentation sets in each locale.
  - The BuildContext should have a property that returns the current LCID that's being built and another that

returns the list of all LCIDs that are being built.
  - Output will be stored in sub folders named with the locale (e.g., bin\Debug\sp-SP\ or Help\bin\sp-SP).
  - Local input will be stored in sub folders named with the locale (e.g., Help\Topics\sp-SP\,

Help\Comments\sp-SP\, Help\Presentation\Style\Content\sp-SP, Help\Presentation\Shared\Content\sp-SP).
  - The default locale's content will be stored at the root;  e.g., Help\Topics\, Help\Comments\, etc. so the

build process for the default locale will work the same way that it does now.
  - Source input can be managed in the same way that version management works.

4. Support dynamic updates to config files at build time.

  - Take all of the <add> elements in the <configChanges> element and move them under a sub element named,

import.
  - Add a new sub element named build that will contain <add> elements for changes to be applied during

builds.
  - replacements for build changes will work the same as for import changes, although they will also allow

string replacement tokens: {0} {1} {2} ...
    - The values of replacement tokens will be defined by attributes on the <add> element: value0="..."

value1="..." value2="..."
    - The values of the replacement attributes are strings that identify the names of public properties on

the current derived implementation of the BuildSettings class during builds; e.g.,

value0="DocumentationSetName"
    - Single-valued properties are converted to strings using Convert.ToString().
    - Properties of System.Collections.IEnumerable implementations are iterated and each element is handled

recursively.  Single-valued elements are converted to strings and IEnumerables are iterated. 

The replacement is executed once for each iteration, recursively.
  - When a build starts the configuration files that will be used are copied to the working directory and

then the build changes are applied.
  - Build assembler will use the config files in the working directory.

5. Support dynamic localization of config files at build time.

  - Add a new sub element to <configChanges> named, localization.
  - replacements for localization changes will work the same as for build changes, including replacement

tokens, expect that replacements are specified as properties of the current BuildContext object.
    - BuildEngine implementations should be able to override a method to create a BuildContext

implementation, such as SandcastleBuildContext, so that properties such as Lcid and Locale are dynamic.
  - When a localized build starts (not the BuildStarting event but the first step of a localized build, which

is executed repeatedly within a single build - once for each locale), the configured localization
    changes are applied, for the current local, to the working copy of the build assembler config files. 

Build assembler must then be executed against the localized copy.  (The original copy must not be

replaced.)

** The explanations above were created before this example.  This example illustrates the _better_ approach

when there are conflicts in the notes above since it's more thought out.

 

- DaveSexton.DocProject.dll.config -

Sandcastle presentation configuration section

<presentations>
  <presentation name="..."

... all of the attributes that currently appear in a presentation's add element ... >
    <options>
      <add name="RunningHeaderText" type="System.String" displayName="Running header text"
           description="The text that will appear at the top of each topic." category="Configuration"

sort="80" />  <!-- the specified type must be convertible to and from System.String

(serialization is not supported) -->
    </options>
    <content>
      <item name="runningHeaderText" value="RunningHeaderText"/>

<!-- value is the name of a dynamic project option here, but it can be any property that

exists on a DocProjectOptions implementation -->
    </content>
    <newProjectWizard>
      <step type="type name" assembly="assembly name"/>

<!-- steps are inserted before the Create Shared Content step in the order that

they are added here -->
    </newProjectWizard>
  </presentation>

  ... more presentations
</presentations>

<configChanges>
  <import>
    ... all of the stuff that is currently in the configChanges section ...

    Note that the default comments folder should be changed from Comments\*.xml to Comments\default\*.xml

since comments should be segregated by locale; 'default' is used since the default locale can be changed.
  </import>
  <build>
    <add name="assign default reference links locale" ....
         xpath="find ResolveReferenceLinksComponent2 locale attribute"
         setAttribute="{0}"
         value0="Locale" />  <!-- Locale refers to a property (that must be added) on the

SandcastleSettings class -->
  </build>
  <localization>
    <add name="assign reference links locale" ....
         xpath="find ResolveReferenceLinksComponent2 locale attribute"
         replaceWith="{0}"
         value0="Locale" required="true" />
    <!-- Locale refers to a property on the SandcastleBuildContext class (this class does not currently

exist; it will derive from BuildContext) -->
    <!-- The required attribute indicates that append should be used if no node is matched by xpath;

although, I'm not sure if I'll be able to determine programmatically whether xpath is selecting an

attribute or an element -->

    <add name="assign localized comments" ....
         xpath="find CopyFromIndexComponent with index name='comments' and select

files='Comments\default\*.xml'"
         replaceWith="Comments\{0}\*.xml"

value0="Locale" />

    <add name="assign shared content items" ....
         xpath="find all of the shared content component's content items that start

with ..\..\Help\Presentation\"
         replaceWith="..\..\Help\Presentation\content\{0}\{1}" value0="Locale" value1="Regex:[^\\]+.xml"
         collection="true" />  <!-- the collection attribute will indicate whether xpath is expected to

return a node collection, in which case this change is applied to each node -->
  </localization>
</configChanges>

April 25, 2008

Sandcastle: Auto-Generated Bibliography

June 11, 2008 Update: This feature is now available as part of the Sandcastle Styles patch and DocProject 1.11.0 RC.

Wouldn't it be cool if you could cite books and web sites while authoring MAML and XML documentation comments and have Sandcastle create a per-topic bibliography automatically?

Someone posted that question in the Sandcastle forums recently and I came up with a solution (in that discussion) that works for the VS 2005 presentation style and both reference and conceptual builds.  Please add your vote to this feature request if you're interested in having this ability in Sandcastle.

This isn't a feature that I'm going to add specifically to DocProject since it requires updating XSL transformations.  You can add it yourself for now though until it becomes a feature of Sandcastle (hopefully :).  Just follow the instructions that I gave in my post in the Sandcastle forum.

Once you make the appropriate updates to the presentation style, here's what you can do to start using this feature in code comments.  (Conceptual usage is similar, although you must add an empty <bibliography/> element somewhere in the root of your MAML, preferably before or after the relatedTopics element.)

First, author some XML documentation comments that contain citations.

<remarks>
  <para>
    The statistics show that Sandcastle is a popular tool on 
    CodePlex <cite>sandcastle activity stats</cite>.
  </para>
  <para>
    DocProject can be used to automate Sandcastle <cite>DocProject</cite>.
  </para>
  <para>
    The top referring sites are similar for DocProject and 
    Sandcastle <cite>sandcastle activity stats</cite><cite>docproject activity stats</cite>.
  </para>
</remarks>

Next, create a bibliography.xml file that provides additional information.  It will be used for all topics that you write in both conceptual and reference builds.

<?xml version="1.0" encoding="utf-8" ?>
<bibliography>
  <reference name="sandcastle activity stats">
    <title>Sandcastle Activity Statistics</title>
    <author>WebTrends</author>
    <publisher>CodePlex.com</publisher>
    <link>http://www.codeplex.com/Sandcastle/stats/view</link>
  </reference>
  <reference name="docproject activity stats">
    <title>DocProject Activity Statistics</title>
    <author>WebTrends</author>
    <publisher>CodePlex.com</publisher>
    <link>http://www.codeplex.com/DocProject/stats/view</link>
  </reference>
  <reference name="DocProject">
    <title>DocProject</title>
    <author>Dave Sexton</author>
    <link>http://www.codeplex.com/DocProject</link>
  </reference>
</bibliography>

Each in-line citation will be replaced with a hyperlink and a number, which corresponds to an entry in the bibliography that is automatically generated at the bottom of each help topic when one or more <cite> elements are detected.

Example Sandcastle Bibliography

April 20, 2008

DocToMaml 1.0 Beta is Now Available

DocToMaml 1.0 Beta is a tool that converts HTML and XHTML help files to Microsoft Assistance Markup Language (MAML) in a batch process.

MAML help files are used by DocProject and Sandcastle to build user documentation in various presentation styles.

DocToMaml provides a console mode and a project-based graphical user interface (GUI) for adding inputs and defining conversion rules to quickly and easily convert HTML files into MAML. Future releases will provide more flexibility for defining rules, but the beta can still be used to greatly decrease the amount of effort that would be required to convert large numbers of HTML files into MAML, compared to doing it manually.

DocToMaml-GUI

Features
Some of the features of DocToMaml are:
  • GUI and console mode.
  • Project-based for quickly saving and loading different configurations.
  • File and folder inputs are supported.
  • Global and input-specific rule sets may be defined.
  • Edit the source HTML of a file input using a WYSIWYG editor for full control over the conversion.
  • Convert a file input in memory so that you can see the result quickly.
  • MAML results for file inputs can be modified in a text box.
  • Batch conversion saves results to disc for all file and folder inputs and generates a conceptual artwork file that can be used by Sandcastle's ResolveArtLinksComponent build component.
  • Hyperlink references to local topics are updated automatically.
  • Preliminary user documentation.
Current Limitations
  • Only the Conceptual MAML document type is supported.
  • Subsections and sub containers are not supported. For example, nested tables and lists are added to the current in-line element, if one exists; otherwise, they are added as new top-level containers.
  • Images appear to be broken in source view, although this has no effect on the output. You don't have to update the paths to broken images since DocToMaml doesn't do anything with image files in the current release anyway.
For help with DocToMaml, see the compiled help file that is available for download on the release page.
April 07, 2008

Article Features DocProject in Microsoft's Dutch .NET Magazine

Jan Schreuder wrote an article on Sandcastle for Microsoft's Dutch .NET Magazine and it's now available on the web.  In it he provides an overview of Sandcastle and some instructions on how to use it.  The article also features a few popular community tools that are available for use with Sandcastle; namely, Sandcastle Help File Builder, AjaxDoc (ScriptDoc) and DocProject.  Jan mentions that DocProject is his preferred tool for automating Sandcastle and cites some of its advantages.  Thanks!

The article is written in Dutch, but I was able to successfully translate it using Google Translator.  First, download the PDF (visit Jan's blog for the link) and then save it as text (check the File menu).  Copy and paste the text into the translator, select Dutch to English and press Translate.  It works well enough to get the gist of the article.

Since the article was written there have been some important changes to Sandcastle and DocProject.  Most notably, Sandcastle is now RTW on CodePlex and DocProject now provides first-class support for conceptual documentation using MAML, which of course wasn't mentioned in the article.

More Information about Sandcastle

For an up-to-date reference check out my (peer reviewed) Sandcastle Help article on CodePlex.  It doesn't go into detail about how to use Sandcastle but it does provide a bullet-point overview and catalogs the different tools and documentation that exist elsewhere on the web.  It's a good starting point.

More Information about DocProject

See my About DocProject article on CodePlex for an overview, screen shots and how to get started.

I also plan to write a few more DocProject tutorials in the future, as well as Sandcastle tutorials for things like how to create a custom presentation style and how to add custom tags, so keep an eye on my blog and DocProject's RSS feed on CodePlex.

April 02, 2008

Auto-Input Protection 2.0 Beta Is Now Available

I've just deployed Auto-Input Protection 2.0 Beta to CodePlex.  (Finally!)

It was changed from a Production release to a Beta release because there are so many breaking changes, although I'm confident that it works since I've been using it on my blog for the last week or two.  I've even fixed a few bugs during that time.

The release page lists the breaking changes and most of the new features.

Here are a couple that aren't mentioned:

  • All of the provider collection elements in the web.config file are now optional.  (It used to be that just the <filters> element was optional.)
  • CrossHatchAutoInputProtectionFilterProvider is now built-in.  Instead of using a diagonal cross hatch each time it selects a HatchStyle value at random, although you can configure it to use a single style if you want.
  • The ProviderHelper class is now public.  It's useful for parsing configuration values from strings into other types.
  • An ASP.NET Web Application project, for testing AIP, is part of the source code provided by the installer.
  • You now have the choice of using the ASP.NET cache or session state to store challenges on the server.  With out-of-process session state, AIP can be used in web farm scenarios.
Security

This release fixes a security flaw that was reported for AIP 1.0.0.  If you find any other flaws please don't hesitate to report them to me so that I can try to fix them.

Note that due to the flaw I've added some new behaviors to help secure AIP; however, it may not be appropriate for immediate use in some scenarios.  A timeout is now implemented that will cause validation to fail if a user does not respond in a timely manner (30 seconds).

If you are using the control on a blog (like me) or any other page that contains reading material or data entry fields, then you can increase the default timeout by setting the ValidationTimeout property on the AIP web control.

Alternatively, although not recommended, you can disable this behavior entirely by setting the ValidationKeepAlive property to true on the AIP web control.  Doing so will cause unused challenges to remain on the server indefinitely, which will increase the amount of memory needed upon each request.  (An unused challenge is one that is requested but never answered.)  If you do decide to disable this timeout (by enabling ValidationKeepAlive) then I recommend setting the PersistenceMode property (new to the AutoInputProtection class) to a value of Session so that at least when a user's session expires their unused challenges will too.  You can set this property easily on the autoInputProtection element in your web.config file:

<autoInputProtection persistenceMode="Session"/>

When you enable session state persistence you should also add the new AutoInputProtectionSessionRequestHandler to the configuration file instead of AutoInputProtectionRequestHandler.  Refer to the docs for more information.

Documentation

The first batch of preliminary docs were built for this release.  Naturally, I used DocProject 1.10.1 RC and Sandcastle so that I could automatically generate reference documentation from my triple-slash code comments and write conceptual documentation using MAML.

The AIP installer merges the MS Help 2 docs (.HxS) into Visual Studio 2005 and 2008 automatically.  The HTML Help 1.x docs (.chm) is provided on the release page as a separate download.

A bit off-topic...

For the DocProject users out there, I've learned a few things about Help 2.x since I built the documentation for AIP and I plan to write a tutorial that describes how to:

  • Add DocSet attributes to your topics so that your documentation appears when filters are applied in Document Explorer, including custom filters.
  • Set the home and default pages.
  • Use the Help Integration Wizard to produce a Merge Module for your .HxS file that can be added to a Setup Project.  The wizard automatically generates the required collection-level files and allows you to specify titles, IDs and custom filters that will be installed automatically.  I'll also describe a few showstoppers that you may run into as well.

There's already documentation online for some of this stuff, but I plan to write this tutorial with DocProject in mind.

In the AIP solution there are two merge modules: one for VS 2005 and one for VS 2008.  Both are referenced by the installer project.  I didn't include the DocProject that I used in the solution simply because AIP was written in Visual Studio 2008 and I was using DocProject 1.10.0 RC, which requires Visual Studio 2005.

March 10, 2008

ContractN on CodePlex

I've just released the first beta version of ContractN, which is a fully managed programming-by-contract framework that uses the CLR's built-in support for aspect-oriented programming (AOP), which provides a way to seamlessly and intuitively describe the required pre- and post-conditions of a type's contract.

I got the idea based on this blog post by Sasha Goldshtein; although, Sasha's implementation and that of ContractN have little in common other than their purpose.  I liked the way Sasha used dynamically-generated code, but other than that I wasn't satisfied having to create the extra stuff that was required to enforce conditions.

Advantages of ContractN

The real benefit of ContractN, IMO, is that there's no overhead in terms of setting up your classes or their consumers, and you don't have to worry about breaking encapsulation or having to create any supporting interfaces or classes.  You just need to derive an object from ContractN.ProgrammingByContract. (It's a shame that this is required though - I hope AOP is eventually introduced as a C# language feature; although, I suspect that the CLR would have to participate as well to alleviate the need for subclassing.)

ProgrammingByContract is an abstract base class that sets up the AOP infrastructure, which is used to intercept method invocations and define conditions as attributes.  After defining a class that inherits from ProgrammingByContract, simply add the appropriate attributes wherever you'd like to enforce certain conditions on the public contract of your type.  Here's a really simple C# 3.0 example:

public class Person : ContractN.ProgrammingByContract
{
  [InRequired, OutRequired]
  public string Name { get; set; }
}

The example illustrates that there are some built-in conditional attributes for basic argument checking, such as InRequiredAttribute, OutRequiredAttribute and ReturnRequiredAttribute, which throw an exception when an argument (or in the case of the latter two, output parameters and a return value) are null.  I plan to add more conditions in the future, and the framework was designed to be flexible enough so that you can define your own conditions as well.

Custom Conditions

To define a new condition, create a class that derives from either PreConditionBaseAttribute, PostConditionBaseAttribute or ConditionBaseAttribute and override the abstract members.  Usage should be intuitive depending upon the implementation that you need, but you can look at the source code for InRequiredAttribute (beta 1) as an example.

Dynamic Conditions

I decided to take a different approach than Sasha in terms of how dynamic conditions are implemented.  I wanted to take advantage of static type checking instead of hard-coding constraints as strings.  PreConditionAttribute and PostConditionAttribute allow you to specify a method name on the decorated type that will be invoked automatically whenever any public method is called.  That also includes constructors and accessors for properties and events.  (Note that these attributes are also valid on a property, constructor, method or event.)  The method signature can vary depending upon your needs; generally it will be a parameterless instance method, though static methods are allowed as well.  The types of arguments that are accepted encapsulate information about a particular invocation.  For example, you could define a method that accepts an argument typed as ContractN.PropertyCall and the method will be invoked automatically whenever a property get or set accessor is invoked, pre- or post-invocation depending upon the attribute used.  Within these methods you can write code that will actually be part of the class that it's meant to constrain, which maintains encapsulation, makes it easy to share code between constraints and allows access to state with compiler support.

Here's a more sophisticated example than above.  It makes use of pre- and post-conditions described above and also implements ad-hoc conditions by providing an implementation of ICondition:

[PreCondition("TestStatic")]
[PreCondition("TestInstance")]
class AdvancedFeatures : ProgrammingByContract, ICondition
{
    #region Public Properties
    public string TestProperty { get; set; }
    #endregion

    #region Methods
    private static void TestStatic(PropertyCall info)
    {
        Console.WriteLine("Pre Static Property: {0}", info);
    }

    private static void TestStatic(MethodCallBase info)
    {
        Console.WriteLine("Pre Static Default: {0}", info);
    }

    private void TestInstance(MethodCallBase info)
    {
        Console.WriteLine("Pre Instance Default: {0}", info);
    }

    public void NoOperation()
    {
    }
    #endregion

    #region IPreCondition Members
    void IPreCondition.Apply(ConstructorCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }

    void IPreCondition.Apply(MethodCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }

    void IPreCondition.Apply(PropertyCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }

    void IPreCondition.Apply(EventRegistrationCall info)
    {
        Console.WriteLine("Pre: {0}", info);
    }
    #endregion

    #region IPostCondition Members
    void IPostCondition.Apply(ConstructorCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }

    void IPostCondition.Apply(MethodCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }

    void IPostCondition.Apply(PropertyCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }
    
    void IPostCondition.Apply(EventRegistrationCall info, MethodCallReturn returnInfo)
    {
        Console.WriteLine("Post: {0}", info);
    }
    #endregion
}

When the above class is used as follows:

AdvancedFeatures features = new AdvancedFeatures();
features.TestProperty = "A test value";
features.NoOperation();

The console output is:

Pre Static Default: ContractN.ConstructorCall
Post: ContractN.ConstructorCall
Pre: ContractN.PropertyCall
Pre Instance Default: ContractN.PropertyCall
Pre Static Property: ContractN.PropertyCall
Post: ContractN.PropertyCall
Pre: ContractN.MethodCall
Pre Instance Default: ContractN.MethodCall
Pre Static Default: ContractN.MethodCall
Post: ContractN.MethodCall

One of the really neat uses for the PreConditionAttribute that I've found is to have ObjectDisposedException automatically thrown instead of having to check for it everywhere.  Here's an example that builds off of the Person class in my first example:

[PreCondition("NotDisposed")]
public class Person : ContractN.ProgrammingByContract, IDisposable
{
    [InRequired, OutRequired]
    public string Name { get; set; }

    private bool disposed;

    private void NotDisposed()
    {
        if (disposed)
            throw new ObjectDisposedException(this.GetType().FullName);
    }

    public void Dispose()
    {
        disposed = true;
    }
}

After the Dispose method is called, public members are no longer accessible to external types.  All calls will result in ObjectDisposedException automatically!

Conclusion

ContractN is more of a hobby project for me now than anything else, but it has potential and I'd really like to see it blossom into something useful.  If you have any suggestions for improvements please let me know and I'll consider them for the next beta release.  (I'm going to establish a timeline on-the-fly as I get ideas for features.)  I have some interesting ideas for new conditions and I hope to blog about them in the future, so don't wander too far off if you're interested...

February 28, 2008

DocProject in the Visual Studio Gallery

Microsoft's Visual Studio Gallery was announced to the public today.  It provides a one-stop-shop for Visual Studio add-ins, packages, item and project templates, macros, controls, coffee makers, toasters, etc.  I think it's a great idea.  Google can only help so much here (have you tried searching for "visual studio add-ins"? ;)

Anyone can register their free or paid products, so if you've got one, let the world know!

I've added DocProject under the Build, Documentation and Web categories.  It has its own little HTML-formatted corner of the site too, which is a cool feature I think.  Take a look at it here.  It already has over 150 unique views!

Currently, the site does not host products so I simply link back to CodePlex (although I love CodePlex so I have no intention of leaving anytime soon).

From my few short discussions with the site's program manager, Anthony, I'd say that they have some good ideas to work with for the future.  Features that'll make locating products and adding new products easier.  So remember to look back occasionally for updates in the next few months.

February 27, 2008

DocProject 1.10.1 Release Candidate Is Now Available

The DocProject 1.10.1 Release Candidate is now available for download on CodePlex.

This release contains some important bug fixes for 1.10.0 RC.  One of which, it was reported, prevented the DocProject Add-In from working at all on some installations of Visual Studio (the Project context menu could not be located).

It also provides built-in support for ResolveExternalLinksComponent, which has been updated to include support for URL mapping provider editors and a built-in database URL mapping provider, among other features and bug fixes. The stand-alone component will be released later this week.

February 27, 2008

ResolveExternalLinksComponent

ResolveExternalLinksComponent is a Sandcastle build component that I wrote to solve a general usability problem that was reported here, by Karl, and to dog food my Hosted Sandcastle Build Component Template.  The component's current behavior was designed with Karl's help.  Thanks Karl :)

Preliminary documentation for the component is here.

Note that the latest release of the component is built-in to DocProject 1.10.1 - a separate download is not required.  When you create a new DocProject or DocSite the component will be included automatically in the Help1.x and Help 2.x reference stacks so that you can begin to use its features right away in your XML documentation comments.

Overview

The original problem, to paraphrase, was that Karl had to keep writing repetitive URLs for <see href="url"/> links throughout his XML documentation comments.  The common URLs he was using pointed to third-party online documentation and each URL shared the same base, so only the relative portion changed between links.  To simplify the process he wanted to write only the relative path each time a link was required, specify the base URL only once, somewhere else, and then have some magic combine them into an absolute path, post-compilation.

As it turns out, a Sandcastle build component was right for the job.

I liked the idea of using IDs instead of hard-coding common URLs throughout the documentation and felt that this would be a feature that I might use, so I developed the component to resolve <see xref="id"/> into hyperlinks using pre-configured ID-to-URL mappings.  For Karl's particular scenario I added support for the vref attribute, which adds additional semantics to the ID so that it will act as the URL itself if a mapping is not defined.

During development I noticed a few opportunities to add more features, which I did, like being able to associate a group of ID-to-URL mappings with any combination of optional attributes.   In particular, a base URL, hyperlink target, auto-generated reference topic ID or import file.  The component allows for any number of mapping groups with any combination of these attributes, allowing for some pretty interesting scenarios.

For example, in one scenario you might define common URLs as IDs for all of your documentation but then override the URLs in one particular topic to point elsewhere, without even having to modify the XML comments in the API.

Another possibility would be to group mappings by hyperlink target.  So, for example, you could set the default target for all mapped links to open in the same browser frame when clicked but then configure a group of links that reference a third-party website to open in a new browser window when clicked.

The component has a graphical editor that works in DocProject and the Sandcastle Help File Builder, which makes configuring it really simple.  It also provides a few DocProject-specific features, such as providing a list of topic IDs that you can choose from, defaulting to the project's settings directory when using an external mappings file, using a relative path to an external mappings file, automatically including an external mappings file as a project item and expansion options for build component stacks in the DocProject Properties window.

What's New In Beta 2

The next release has a few bug fixes and some neat features that I'd like to mention here.  You can read about additional updates on the release page.

First, I've added support for resolving IDs to URLs that are stored in a database.  (See the section on URL Mapping Providers below for more information.)

You can now use xref and vref attributes on seealso tags as well; although, due to a current limitation in Sandcastle, the target attribute will be ignored.  However, if Sandcastle eventually supports the target attribute on seealso tags then my component should work without having to be rebuilt.

The editor now provides the ability to edit the base and target attributes on the component element.

In DocProject, if you configure an external mappings file then it will be included as a project item automatically.  Also, you can use the open file dialog to specify the name of a file that doesn't actually exist and it will be created when the editor is closed.  The path of the file will be made relative to DocProject's working directory.

URL Mapping Providers

The common way to define a group of mappings is to add them to the component's XML configuration using the mappings and url elements.  The class that parses url elements is, XmlUrlMappingProvider, and as you may have already guessed, it's a specialization of the abstract base class, UrlMappingProvider.  Here's what a common configuration might look like:

<component type="DaveSexton.Sandcastle.ResolveExternalLinksComponent"
           assembly="C:\...\DaveSexton.Sandcastle.dll">
  <mappings>
    <url id="home" href="http://davesexton.com" />
    <url id="contact" href="http://davesexton.com/Contact" />
  </mappings>
</component>

You can create your own custom providers as well.  To use them simply specify the full type name and path to the assembly using the type and assembly attributes, respectively, on the mappings element.  The outer XML of the mappings element will be supplied to your provider as raw text.

URL mapping providers can have a UI that will be displayed in the component's editor to easily modify the provider's configuration.  The built-in XmlUrlMappingProvider uses a simple DataGridView for editing the url elements.

DatabaseUrlMappingProvider

DatabaseUrlMappingProvider is a new provider as of DocProject 1.10.1 and the ResolveExternalLinksComponent Beta 2 release.  It allows you to define ID-to-URL mappings in a database table instead of the inner XML of a mappings element.

This provider could be useful in scenarios where you already have URLs stored in a database or when you have a large number of URLs that must be shared between multiple tools, whether related to documentation or not.

A custom editor that allows you to configure the provider is embedded into the component's editor.  With it you can easily specify the data provider, connection string, query, parameters and other settings all in one place:

DatabaseUrlMappingProvider Editor

Here's an example configuration that might be used with this provider:

<component type="DaveSexton.Sandcastle.ResolveExternalLinksComponent"
           assembly="%DocProjectBuildPath%\DaveSexton.Sandcastle.dll">
  <mappings type="DaveSexton.Sandcastle.DatabaseUrlMappingProvider"
            topic="M:SomeNamespace.SomeType.DocumentedMethod(System.String)">
    <connection provider="System.Data.SqlClient">
      Data Source=DEV1\SQLEXPRESS;Initial Catalog=Docs;Integrated Security=True
    </connection>
    <query>
      SELECT ID, HREF FROM UrlMappings WHERE TopicID = @TopicID;
    </query>
    <parameters>
      <parameter name="@TopicID" source="TopicID" />
    </parameters>
  </mappings>
</component>

The component can use any of the built-in Framework data providers (e.g, Odbc, OleDb, SqlClient) or even plug-ins such as Oracle.

Parameterized textual queries and stored procedures are supported, but only the first result set is used.  By default the provider will look for ID and HREF columns, but you can specify different names.

Parameters can have hard-coded values or they can be mapped to a specific source.  In the example above the value of the topic attribute will be used as the value for the @TopicID parameter.  The other acceptable values for the source attribute are: BaseUrl, File and Target, which also correspond to attributes on the mappings element.  To define a specific value for a parameter use the value attribute instead of source.

February 13, 2008

Sandcastle Build Component Templates 1.1 Released

The Sandcastle Build Component Templates 1.1 release provides two C# Item Templates that you can use to easily create custom Sandcastle Build Components in Visual Studio 2005 and Visual Studio 2008.

In this blog post I intend to provide some details about the templates, with more focus on hosted components and DocProject's APIInstructions for template installation and usage can be found here.

Basic Sandcastle Build Component Item Template

The Basic Sandcastle Build Component item template is a multi-file Visual Studio Item Template, written in C#, that provides a working example of a custom build component that can be used for command-line builds or with editor support in automation tools such as DocProject and the Sandcastle Help File Builder.  It comes with a built-in graphical editor and a dynamic sub property for the DocProject Properties window, which can be shown by expanding the component in a build component stack (for more information about the stack properties, see How To Use Third-Party Sandcastle Components in DocProject).

Hosted Sandcastle Build Component Item Template

The Hosted Sandcastle Build Component item template is the same as the basic template, but it also provides direct access and a working example of how to gather information from DocProject's API, thus DocProject is required in order to compile the build components that are created based on this template.  Although, DocProject isn't actually required to use the components if you provide a fall-back initialization path.

Initialization Paths

The template's Configuration class provides two initialization paths, one of which is a fall-back that's automatically invoked if the DocProject host API is not detected.

Note: The host API is never available during builds - it's only available when the component's editor is displayed.

Fall-back initialization works since the template depends upon just-in-time (JIT) compilation.  If you properly encapsulate the use of host interfaces, as the template does out-of-the-box, then you can control the initialization path of the component's Configuration class.

The template's Host class is used to encapsulate host interfaces and services, and it provides a way to safely check, at runtime, whether a particular host is reachable.  Support for DocProject's host API is built-in to the template.  All other hosts will use the fall-back initialization path automatically unless you add support for them to the Host and Configuration classes.  The only requirement is that the host invokes your component's editor by calling its EditValue method and passes in an IServiceProvider implementation that can be used to obtain a reference to one or more of the host's services.  (On a side note, the value argument should be set to a System.Xml.XPath.IXPathNavigable implementation that's positioned on the outer XML of the component's configuration, which includes the <component> element itself.)

Modifying Project Options

The template gathers read-only data as an example and does not actually modify any build settings or project options at runtime.  If you want your component implementation to write to the API, then make sure there's a clear relationship between your component and the changes that it's making.

For example, a proprietary AuthorBuildComponent changing the value of the Generate root API topic setting when edited will probably not be obvious to end-users.  But it might be appropriate for the same component editor to add external sources if, for example, it were to provide a way for an end-user to create a list of associations between sources and author names, much like the Version management dialog does with version information.   In this case, the component's editor could save the input-to-author mappings as its inner XML configuration data and add the external sources as a convenience for the user.

Hosted Components as Plug-Ins

Hosted build components are not meant to be a plug-in mechanism for DocProject, per se, but instead are a plug-in mechanism for Sandcastle.  DocProject's API is exposed merely to provide a more user-friendly and integrated experience when editing Sandcastle build components in DocProject.

If you want to create a true DocProject plug-in, then create a custom build engine provider or a build process component, which can provide control over all aspects of DocProject, including the entire build process, engine settings, project options, tool bars, tool windows, menus, custom wizard steps in the New Project Wizard, etc.

DocProject's API for Hosted Components

With the recent release of DocProject 1.10.0 RC, a new interface was added that would provide build components access to DocProject's API, for their graphical editors.  The API can be used to gather information about the environment in which the component's editor is being hosted.  The component can then provide automatic configuration and apply appropriate editor defaults, simplifying configuration for end-users while also adhering to the project paradigm that DocProjects and DocSites use to store their settings.

The purpose of the gateway interface, IDocProjectHost is to provide components with access to DocProject-specific interfaces and types for the context in which the component's editor is being hosted.  Here's the interface's definition:

public interface IDocProjectHost
{
    IDocProject Project { get; }
    BuildSettings Settings { get; }
    DocProjectOptions Options { get; }
    IBuildEngine Engine { get; }
    IEnvironmentHost EnvironmentHost { get; }
    bool RunningInVisualStudio { get; }
}

The Hosted template gets a reference to an IDocProjectHost implementation through the service provider that's passed to the editor's EditValue method.  If the host is available then the Configuration class's InitializeForDocProject() method is invoked instead of the fall-back Initialize() method (see above for information about initialization paths).

The InitializeForDocProject method uses the types and interfaces exposed through IDocProjectHost to gather information, which it then stores in private fields.  The fields back properties that are used by the template's EditorControl class.  They are typed as nullable primitives, such as System.String and bool?, so if the values are never set then they'll remain null, in which case the EditorControl will substitute them with "Unavailable" for its read-only display.

Developers are free to remove this code (actually, it's recommended) and replace it with appropriate code for gathering information that's required by the component being developed.  However, the same pattern can be used to safely encapsulate data that can be initialized with or without a host API.

Note: The configuration fields must not be typed as host-specific types or interfaces if you want to be able to use your components in other environments, without the host API installed.

Conclusion

With the release of DocProject 1.10.0 RC and the corresponding Hosted Sandcastle Build Component Item Template 1.1, developers can now have seamless integration in DocProject for their custom build components' editors, providing a more robust platform on which to configure and build documentation while potentially simplifying configuration for end-users through automation and inferred default settings.