November 20, 2007

DocProject 2008 Beta 2 Released

DocProject 2008 Beta 2 has been deployed to CodePlex.

  • Compatible with Visual Studio 2008 RTM (tested against Team Suite Trial).

  • Compatible with Visual C# 2008 Express and Visual Basic 2008 Express.

  • Supports side-by-side installation with the DocProject 1.9.0 RC for Visual Studio 2005.

  • Built using the same codebase as the 1.9.0 RC, with updates to support Visual Studio 2008, a few subtle changes that will also be included in the 1.10.0 RC, and some bug fixes.

Bug Fixes
  1. Topic Management Error: Value cannot be null
  2. Deleting a comment section does not mark the content item as dirty, so if it's the only change that's made it's not committed when the dialog is closed.
  3. The Cancel and Back buttons cause control validation on the Wizard Form, but they shouldn't.
  4. DocProjectEnvironment.InstallVSExtensions adds the same commands and tool windows to the commands and tool window collections even if they're already present.
  5. DocProject Properties window may be added to the wrong Project command bar, depending upon the environment.
Known Issue

The Build Component Stack configuration options display Load Error messages. This issue is related to a problem in Visual Studio 2008 that has yet to be resolved; however, if you manually add Sandcastle's assemblies to the GAC then you will not experience this issue.

See my blog post about this issue for more information.

Side-By-Side Installation

DocProject 2008 Beta 2 will work side-by-side with DocProject 1.9.0 RC, although you must update new and existing 1.9.0 DocProjects and DocSites so that they use a specific version of the DaveSexton.DocProject assembly.

To ensure that your 1.9.0 RC projects continue to work, follow these steps after installing DocProject 2008 Beta 2:

  1. Open an existing 1.9.0 RC DocProject or DocSite in Visual Studio 2005.
    1. If you need to create a new project instead then continue to the next step after completing DocProject's New Project Wizard.
  2. Expand the References folder in Solution Explorer.
  3. Right-mouse click the DaveSexton.DocProject assembly and select Properties.
  4. Set Specific Version to True. The version number, which is read-only, should automatically change from 2.0.0.0 to 1.9.0.0.
  5. Save the project.
If you do not follow these instructions then you will get an error when you attempt to build the project that states, The specified type is not a valid BuildProcessComponent implementation.
November 16, 2007

DocProject 2008 Beta 2 - AssemblyResolve Showstopper

April 27, 2008 Update: The problem is with Type.GetType forcing a complete JIT when executing in VS 2008 (it is used by InstantiateProviders).  I've submitted a new bug report that provides a much cleaner solution to reproduce the problem: https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=340664

Nov. 20, 2007 Update: I've narrowed the problem down to a call to System.Web.Configuration.ProvidersHelper.InstantiateProviders, which tries to resolve all dependencies at the time of invocation even if they are not being referenced in any executing code.  If assembly resolution must be deferred (as is the case in DocProject) then this presents a problem because there will not be an AppDomain.AssemblyResolve event handler registered to help the framework resolve the assembly.  However, this behavior only occurs when VS 2008 is hosting the executing code.  It does not occur in VS 2005 or in a console application built on the .NET 3.5 framework, for example.

I've submitted feedback to the Sandcastle team to have them GAC the required Sandcastle assemblies to prevent the issue in this blog post from occurring.  For now though, you can manually GAC the BuildComponents.dll, BuildAssemblerLibrary.dll and CommandLine.dll assemblies using: gacutil /i assembly_name.dll

I will be releasing DocProject 2008 Beta 2 shortly.

I've finished working on DocProject 2008 Beta 2, however, there is an issue with the new build component stack properties that has prevented me from releasing it: When the stack properties are expanded, all of their components are replaced with load error messages.

If you feel that this issue should not be preventing me from releasing the product then please let me know and I'll consider removing the expandable stack properties for now in 2008 Beta 2.

More Details

The problem is that an assembly cannot be delay-loaded in certain situations while a program is being hosted by VS 2008 since the AppDomain.AssemblyResolve event is not being raised.  The same code works in VS 2005 and using .NET 3.5 in a console app, but it just doesn't work when executed by a VS 2008 add-in.

It appears to be the same bug in VS 2008 that I reported back in August, but at the time I had discovered a work-around that was successful because the assembly was being loaded in another AppDomain.  Unfortunately, it won't work this time since the components must be loaded in the default AppDomain.

I've submitted new comments (starting at 11/16/07) in the feedback with a project that reproduces the bug attached, so I'm hoping that someone will post a workaround (I assume that it's way too late to get fixed in VS now).  Download the attachment from my website if you think you can help.  The instructions are in my feedback post :)

Here's the output that the example produces in the debug window.  Notice how the console version of the program appropriately raises the AppDomain.AssemblyResolve event, where as the VS add-in version does not.  The exception is actually not the problem - it's to be expected since I didn't add any code to actually resolve the assembly.  (The point is that VS 2008 doesn't even give DocProject a chance to resolve the assembly).  And BTW, both hosting scenarios in my example call into the exact same code.

The example was developed on the latest release of the Windows Server 2003 VPC for Visual Studio 2008 Team System Beta 2.

CONSOLE OUTPUT:

'ConsoleApp.vshost.exe' (Managed): Loaded 'C:\Documents and Settings\Administrator\My Documents\Visual Studio 2008\Projects\TestAddIn\ThirdPartyLibrary\bin\Debug\ThirdPartyLibrary.dll', Symbols loaded.
AssemblyLoad: ThirdPartyLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8fd3dad2bd735269
*********** Loading type...
AssemblyResolve: ThirdPartyLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8fd3dad2bd735269
A first chance exception of type 'System.IO.FileNotFoundException' occurred in PlugInLibrary.dll
The thread 0xbfc has exited with code 0 (0x0).
The thread 0xd08 has exited with code 0 (0x0).
The program '[1180] ConsoleApp.vshost.exe: Managed' has exited with code 0 (0x0).

VISUAL STUDIO OUTPUT:

'devenv.exe' (Managed): Loaded 'C:\Documents and Settings\Administrator\My Documents\Visual Studio 2008\Projects\TestAddIn\ThirdPartyLibrary\bin\Debug\ThirdPartyLibrary.dll', Symbols loaded.
AssemblyLoad: ThirdPartyLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8fd3dad2bd735269
*********** Loading type...
A first chance exception of type 'System.IO.FileNotFoundException' occurred in PlugInLibrary.dll
'devenv.exe' (Managed): Loaded 'C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\Microsoft.VisualStudio.TeamFoundation.Build.dll'
AssemblyLoad: Microsoft.VisualStudio.TeamFoundation.Build, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
The program '[3292] devenv.exe: Managed' has exited with code 0 (0x0).

November 09, 2007

DocProject Tips and Tricks I

As you probably already know, DocProject is kind of weak in the documentation department (that is, in its own API documentation :)  I plan to address this issue for the first RTW, after I include first-class support for generating conceptual content and after Sandcastle's first RTW.  If you are new to DocProject, the articles on CodePlex should provide more than enough information to get you started.

In a series of blog posts, with this post as the leader, I'm going to provide some supplementary information that you may find useful.  Tips and tricks that will add value to DocProject and allow you to make the most of it.  Code examples can be found at the end of posts and will have names like, Example #.

Community Tricksters

As the author, I probably know more about DocProject than anyone.  But as an end-user, I haven't taken advantage of the full range of its capabilities yet - including things like using DocProject with Team Build, building AJAX documentation or customizing Help 2.x output.  Therefore, I'd be happy to accept tips and tricks from the community that I can include in this series.  If you have ideas please submit them as blog comments or email them to me and I'll consider including them in my posts.  Thanks!

General-Purpose Tips

Here are some tips for DocProject that apply to all types of templates and build engines:

  1. If you do not want to include build output as project items, and you don't want to be prompted after you build a project for the first time, then set Include Project Output Dialog | Don't ask me this again to True in the DocProject Properties window before you build.  You can leave the default values for the two related options, which should both be False.  (Note that the Apply to all default option is not used when Don't ask me this again is True.)
  2. You can insert dynamic build steps using your project's Build Process Component (see Example 1 below).
  3. Use an external or third-party Build Process Component:
    1. Open the DocProject Properties window (right-mouse click your project in Solution Explorer and select, DocProject Properties).  Visual Studio Express users must use the DocProject External UI.
    2. Locate the Extensibility | Process component type name option.
    3. Enter an assembly-qualified type name (the assembly must be located in the GAC or your project's output folder).
    4. Alternatively, you can enter a full type name and an absolute or relative assembly path, separated by a vertical bar; e.g., MyNamespace.MyType|..\anotherproj\bin\Debug\anotherproj.dll
DocSites Tips

These tips apply to the DocSite Templates only:

  1. To change the DocSite logo simply replace the Images\Logo.jpg file with a file of your own.  To use a different file name, change the value of the ImageUrl attribute on the <asp:Image ... id="logoImage" ... /> element in the Controls\DocSiteHeader.ascx file.
  2. To show rank information in DocSite search results, for the purpose of fine-tuning the search index, open the App_Themes\BasicBlue\DocSiteSearch.skin file and set Visible="True" on <asp:Label ... SkinId="searchResultRank" ... />.  When performing a search the rank value will appear before the title of each matching topic.  If you already have search results in your browser just refresh the page.
  3. To change the DocSite title, company name and copyright, set the value of the AssemblyTitle, AssemblyCompany and AssemblyCopyright attributes, respectively, in the Properties\AssemblyInfo.(cs|vb) file and then restart the web application.  (The DocSite caches these values in memory upon the first request to the site.)
Sandcastle Tips

This section provides some useful tips for projects that use the Sandcastle or Sandcastle/Deployment build engine providers.

  1. Set the Help 1.x Configuration File Name and Help 2.x Configuration File Name options to the same values and DocProject will only run Build Assembler once to build both Help 1.x and Help 2.x output.  The cost of doing this is that reference link types in one of the help output formats might not be valid (this was the main reason for splitting the Sandcastle configuration file into two distinct files in the first place.)
  2. Use XML documentation files generated in the API Topic Management dialog for IntelliSense by adding a short code snippet in your DocProject's Build Process Component (see Example 2 below).
  3. In your Build Process Component, you can get access to all of the Sandcastle-specific project options and APIs.
    1. Add a reference in your DocProject or DocSite to the DaveSexton.DocProject.Sandcastle.dll assembly, commonly found at C:\Program Files\Dave Sexton\DocProject\bin\.
    2. Check out the code in Example 3 below, which illustrates how to access the custom Sandcastle APIs that are provided by DocProject.
  4. Insert images into your documentation using the XML documentation editor in the API Topic Management dialog:
    1. In design mode, place the text cursor where you want to insert an image.
    2. Click the InsertPictureHSbutton on the editor's tool strip or right-mouse click and select Insert > Picture.
    3. In the Picture dialog enter the full path and name of an existing image or click the Browse... button to locate one.
    4. You can enter a local file path or even a web address such as, http://example.com/someimage.jpg.  Click OK and the image is inserted.
    5. After you are done authoring your documentation, click OK on the API Topic Management dialog and DocProject will automatically import the image into your project's Help\Art folder.  It will also rebase the reference in the XML documentation to ..\Art\{image name}.
      1. Note: If you enter a web address then DocProject will copy the image from your web browser's cache; otherwise, the file is copied from the location that you specify.
    6. Note: The next time that you view the XML documentation in the API Topic Management dialog, the path will be rebased so that it's rooted, for display purposes only.  The path in your actual XML documentation on disc will remain relative, however.
  5. Filter out <Module> topics when the Documentation scope option is set to Complete by adding a dynamic regular expression filter:
    1. Open the API Topic Management dialog.
    2. Select the Topic Filters tab.
    3. Select the Regular Expression tab.
    4. Enter the following into the Pattern text box: \<Module\>
    5. Click the Save button.  The Saved Filters tab will be shown automatically and your filter will appear in the list.
    6. For the new filter, ensure that the Build column is checked and that the Include column is unchecked.
    7. Enter a short description of this filter into the Memo field (optional).
    8. Click OK on the API Topic Management dialog to commit your changes.
C# Code Examples

The following examples illustrate points made in the information above.  They are referenced by number.

Example 1:

The following example shows how to inject a dynamic build step immediately before the Help 1.x compiler step using your project's Build Process Component.  The step will be executed in full and partial builds and will run on a background thread.  It outputs some text in the build trace window:

Step # of ##: Hello

World!

To use this example, replace the BuildStarting method in your project's BuildProcess.(cs|vb) file.  (Keep the existing code though).

   1: protected override void BuildStarting(BuildContext context)
   2: {
   3:   // existing code goes here 
   4:  
   5:   base.InsertBeforeBuildStep("Compile Help 1.x", "Hello", true, false, true,
   6:     delegate(BuildContext stepContext)
   7:     {
   8:       stepContext.TraceLine("World!");
   9:     });
  10: }

Example 2:

The following example will overwrite the source XML documentation file (that contains your code comments) with the merged XML documentation file (that contains both your external documentation and the source code comments) after each help build completes.

To use this example, replace the BuildCompleted method in your project's BuildProcess.(cs|vb) file.  (The existing code should be placed at the end).  You must also add a using System.IO declaration at the top of the file.

   1: protected override void BuildCompleted(BuildContext context)
   2: {
   3:   TraceLine();
   4:   TraceLine("Overwriting XML documentation files in source projects..."); 
   5:  
   6:   IDocProject project = context.Engine.Project;
   7:   BuildSettings settings = context.Engine.Settings; 
   8:  
   9:   foreach (ISourceProject source in project.Sources)
  10:   {
  11:     string target = source.Output.XmlDocumentationFile; 
  12:  
  13:     FileInfo autoGenFile = new FileInfo(Path.Combine(
  14:       settings.WorkingXmlDocumentationPath, 
  15:       Path.GetFileName(target))); 
  16:  
  17:     if (autoGenFile.Exists)
  18:       autoGenFile.CopyTo(target, true);  // overwrite
  19:   } 
  20:  
  21:   TraceLine("Done."); 
  22:  
  23:   // existing code goes here
  24: }

Example 3:

You can access Sandcastle-specific project options and APIs in your Build Process Components by adding a reference to the DaveSexton.DocProject.Sandcastle.dll assembly.  The following example shows how to obtain the SandcastleBuildEngine instance, and from that retrieve the SandcastleSettings and SandcastleProjectOptions:

   1: SandcastleBuildEngine engine = (SandcastleBuildEngine) context.Engine;
   2: SandcastleSettings settings = engine.Settings;
   3: SandcastleProjectOptions options = engine.Options;

The context is an argument that is passed to all of the four main Build Process Component overrides: BuildStarting, BeforeExecuteStep, AfterExecuteStep and BuildCompleted.

November 07, 2007

DocProject 1.9.0 RC Deployed

The DocProject 1.9.0 Release Candidate is now available on CodePlex.  I'm still working on updating the wiki documentation though.

Thanks for your patience with this one.  I invested more time than I probably should have in additional features that weren't immediately necessary, but hey, this project is fun for me and some times I can't seem to come to a logical stop :)

New Features in DocProject 1.9.0 RC

  • Compatibility with the Sandcastle October 2007 CTP.
  • Preliminary support for conceptual content using Sandcastle's conceptual configuration file. (The conceptual build process has been integrated although it hasn't been tested and DocProject does not automatically generate a conceptual TOC yet.)
  • Sandcastle's build component stacks are now available as properties in the DocProject Properties window and the DocProject External UI, with support for hosting custom build component editors that can either show a modal dialog, a drop down editor or subproperties.
  • Framework and project reference link types are configurable as subproperties of the build component stacks and can be set by simply choosing one of the following values from a drop-down list: MSDN, Index, Local or None.
  • DocSite Templates include four new buttons under the Contents tab: Save, Bookmark, Email and Print. The latter two are supported in IE6, IE7, Firefox and Opera, but the former two are only visible in IE. Note that Save does not currently download supporting files such as JavaScript or style sheets though.
  • DocSite performance generating the full-text search index has been greatly improved.
  • Version Management Dialog: Create documentation that includes version information by referencing external assemblies and reflection files, and then simply enter custom version names (DocProject uses Sandcastle's VersionBuilder program in the background.)
  • ChmBuilder is used exclusively to build the supporting Help 1.x files (WARNING: The .HHC file format has been changed. Beware if you were modifying it to include custom topics.)
  • Both the Topic Designer and API Topic Management dialog now remember their last saved position, size, and the positions of splitters.
  • Edit all content item documents using the Topic Designer instead of just the shared_content.xml file. All of the available documents are listed in the Create Shared Content step of the New Project Wizard as well.
  • Several new API Topic Management dialog features:
    • Improved performance when applying filters.
    • Topic tree nodes now use icons that reflect access type, as do the tooltips.
    • Filter topics by API accessibility or in combination with API types; e.g., protected fields, private interfaces, public or protected methods and delegates, etc.
    • Save filters and load them later or have them applied automatically during builds (dynamic filters).
    • Author external XML documentation in the appearance of the selected presentation style, with special styles for XML documentation tags such as see, c, code, paramref, etc.
    • New XmlTreeView project and control, which supports binding to an IXPathNavigable data source and allows drag/drop internally and externally (although internal drag/drop support is not currently enabled. This feature is planned for conceptual content in a subsequent release).
    • Drag and drop API elements from the tree into the XML documentation editor to generate see links automatically (works in both source and design mode).
    • The appropriate XML documentation sections for each specific API type (.e.g, summary, remarks, return, value, etc.) are listed automatically in the XML documentation editor, including individual param and typeparam tags for methods and delegates.
    • Add additional documentation sections that support multiple instances (e.g., exception, permission) by selecting them from a drop-down list. The list only shows those sections that are appropriate for the selected topic.
    • Quickly create custom XML documentation sections in the editor by entering a name and pressing enter, or register them as built-in sections in DocProject's configuration file. See this blog post for information on how to use custom tags in Sandcastle's XSL transformations.
    • The XML documentation editor automatically cleans up empty sections from the external documentation when it's saved.
  • Build Process Components can now take advantage of more of DocProject's API and now have the ability to inject dynamic build steps, pass delegates to DocProject's API and register event handlers.
  • Documentation now includes a root Namespaces topic to which the XML documentation's project summary is applied.
October 27, 2007

DocProject 1.9.0 RC Preview

The DocProject 1.9.0 Release Candidate will contain several bug fixes and some exciting new features (well, exciting to me anyway :). In this blog post I'm going to describe some of the new features and show a few screen shots, like usual.

Miscellaneous Features and Bug Fixes

For the sake of time and keeping this blog post short (well, shorter than I could make it), I'm going to first list a few of the important features that I feel deserve to be mentioned, even if I'm not going to elaborate:

  1. The time it takes to generate the DocSite's full-text search index has been greatly reduced - to the point where it should actually be useable now.  In my testing, no more than 10 minutes for over 10 000 topics on a regular desktop computer.  I still plan to improve performance even more though and will develop that SqlSearchProvider very soon.
  2. The DaveSexton.DocProject.DocSites library no longer requires full trust.
  3. DocSites now have Save, Print, Email and Bookmark buttons in the Contents bar to act on the selected topic.  The Print and Email buttons work in IE6, IE7, Firefox and Opera.  The Save and Bookmark buttons only work (and are only visible) in IE6 and IE7.
  4. Build Process Components can now use more of DocProject's API, including the use of cross-AppDomain delegates and event handlers and they are even able to insert dynamic build steps that are actually build steps.   I'd like to blog about this separately though.
  5. The Sandcastle plug-in now uses the ChmBuilder program to generate the supporting Help 1.x files.  Localization has not actually been implemented yet, though I plan to address that soon.
  6. The Topic Designer (shared content) dialog now displays all of the files that have Content/<item> semantics, and they are listed as Content Documents (this includes documents in Shared\Content\ as well).  Selecting a document will automatically list all of its items underneath, and from their the dialog works the same as usual when an item is selected.  By default, shared_content.xml and header are selected, like usual.
  7. Bug fixes galore.  (Though, it was more bugs due to major refactoring in this release than existing bugs from previously releases :)

And now for some elaboration...

API Topic Management Dialog

Lots of new stuff and refactored code here.  I'm not really sure where to begin, so I guess I'll just start off by saying that the API Topic Management dialog now saves its last location and size, and the position of the splitters (and so does the Topic Designer).  Now, let's continue on with some of the more obvious and important changes.

image

Figure 1: API Topic Management Dialog - Tree View and Regular Expression Topic Filter

Tree View

As you can see from the image above, the tree now shows general topics that are defined by the presentation style's doc model (e.g., All Members, Methods, Fields, Properties, etc.).  Theses types of topics cannot be filtered with the checkbox; however, uncheck them and all child nodes will be unchecked automatically.

You may also notice in the image that the icons are now synchronized to the API access levels of each topic.  For example, in the image above, AI.Cells.Enzyme.acids is a private field, which has an icon with a lock.  The tool tip will also indicate that the topic is a "Private Field".

The +/- button above the tree will, when clicked, collapse all nodes if any one is currently expanded, and will expand all nodes if they are all collapsed.  There are other buttons that are currently hidden because they have to do with conceptual topics.  More on that later...

It's probably also worth noting that the performance of the tree when loading data and finding/applying topic filters has been improved.

Topic Filters

Figure 1 also depicts the improved Topic Filters tab and some of its new features.  But let's just start with the basics...

Regular Expression Filter

The options for the Regex filter are now displayed as checkboxes, simply for convenience since there is now more room in the GUI due to the filter types having their own individual tabs.  Also, the pattern can now be entered as multi-line, although since new lines will not match any topic names, entering multiple lines (to improve readability of your patterns) will cause the Ignore Pattern White Space option to be checked automatically.

Categories Filter

The categorical filter has also been improved, for performance and functionality.  It now supports filtering by API access levels, in addition to API category types.  For example, you can now filter by protected fields, or private interfaces and structures.

image

Figure 2: API Topic Management Dialog - Categories Filter

If nothing is checked under Accessibility then it is simply not taken into account when matching.  In other words, any access type is a match - the same as if all access types are checked.

You can also choose an access level without choosing an API type and it will match all APIs with the selected access level.  Choosing more than one access level is an OR operation, just like with API types.

To match protected internal members you must check protected and internal since it's treated as a distinct access type.  In other words, DocProject does not treat protected internal members as just being protected, or internal.  One caveat is that searching for protected internal members will also explicitly match protected-only and internal-only members as well.

Saved Filters

Saved filters are a new addition in 1.9.0.  It allows you to save Regex and Category filters so they can be loaded and applied quickly in the future or, more importantly, applied automatically during each build.

image

Figure 3: API Topic Management Dialog - Saved Filters

You can get filters into the list by clicking the Save button that appears on each of the filter tabs (see Figures 1 and 2).  Alternatively, you can directly edit the XML file in which they are saved, located at Help\dynamicFilters.xml.  The file does not exist until you add a filter using the dialog and then click OK, but you can create the file yourself if you want (see Figure 4 below).

You can also import filters from a Dynamic Filters XML file using the Import... button (see Figure 4 below).  Imported filters are appended to the list and do not replace any existing filters.

Memo is a place to comment on filters.  You can edit the memo directly in the grid after you save a filter and it will be persisted along with the filter list when you save the changes to the API Topic Management dialog.

Clicking a Load button will simply load the corresponding filter into the appropriate filter tab (i.e., Regular Expression or Categories) and will then show the tab and apply the value of the Include column to the Include matching topics Search Option.  The filter will not be applied, however.

The Apply button will immediately apply the filter to the tree using the current Search Options and the value in the Include column, but will not load the filter first.

You can delete the selected filter by pressing Delete on the keyboard or by clicking the delete button (with the red X).

The arrows are used to order the filters, which is important because all filters that have a check mark in the Build column are applied in order, top-down, during a help build.  The Apply all filters during builds option will either check or uncheck the Build column for all filters so that they have the same value as the option and each other.  The Apply All button will apply all filters to the tree view immediately, top-down, but without regard to the Build column.

The Include column indicates whether matching topics are included or excluded, just like the Include matching topics Search Option.

These features together provide a powerful way to filter topics.  For example, in Figure 3 above the 3rd filter excludes all private fields, but then the 4th filter ensures that one private field in particular, acids is included again since the Include column is checked.  The 4th filter will be applied after the 3rd filter, since it appears after the 3rd filter in the list, so the acids field will end up being included after all filters have been processed on that topic.

The schema of the XML file that stores the filters is simple, so I'll just give an example file that was created from the filters in Figure 3:

<?xml version="1.0" encoding="utf-8"?>
<filters>
  <filter type="DaveSexton.DocProject.Sandcastle.TopicManagement.RegexTopicFilter">
    <include>False</include>
    <pattern>Carbon$</pattern>
    <memo>Get rid of any topic that ends with "Carbon".</memo>
  </filter>
  <filter type="DaveSexton.DocProject.Sandcastle.TopicManagement.RegexTopicFilter">
    <include>False</include>
    <options>IgnoreCase</options>
    <pattern>^Startup$</pattern>
    <memo>Get rid of the Startup classes in each assembly.</memo>
  </filter>
  <filter type="DaveSexton.DocProject.Sandcastle.TopicManagement.CategoryTopicFilter">
    <include>False</include>
    <memo>No private fields.</memo>
    <types>Field</types>
    <access>Private</access>
  </filter>
  <filter type="DaveSexton.DocProject.Sandcastle.TopicManagement.RegexTopicFilter">
    <include>True</include>
    <pattern>^acids$</pattern>
    <memo>Reinclude the "acids" field, even though it's private.</memo>
  </filter>
</filters>

Figure 4: Example Help\dynamicFilters.xml file

Note that the type attribute will accept assembly-qualified type names so that you can define your own custom filters; however, I've not actually tested custom filter types yet so they may not actually work, but I plan to support them in a subsequent release.

The element names for a filter's options are defined by the filter itself, although memo, include and build (which is not shown in the example because it defaults to True) are built-in to the base type because they have special meaning to DocProject.

XML Documentation

In 1.9.0 there are some new features to help make authoring documentation easier.  But before I begin on that, I'd like to lead in with some info about where DocProject currently stands on handling conceptual content since it has had a profound effect on 1.9.0.

Conceptual Content

The next release of Sandcastle might be available on Monday (the 29th of October), and you can probably expect DocProject's release to follow within a few days. One thing that I'm really excited about is that the Sandcastle team will be shipping their internal conceptual configuration file for the Build Assembler program, which should allow me to enable the long awaited first-class support for conceptual content in DocProject.  If I can include support in 1.9.0 without too much delay then I certainly will, but if that's not the case then expect it in 1.10.0.

The Tree View and XML Documentation

In preparation for conceptual content, I've refactored the code some more and made some assumptions about how the process will be integrated.  Also, I've included my new XmlTreeView control with DocProject 1.9.0 RC (it's being used in the API Topic Management dialog now).  It supports drag/drop to reorder nodes (to be used for conceptual content only, once it's supported) and to drag nodes onto the XML documentation editor, which automatically creates a <see> link (that is supported in 1.9.0).  Also, the XML documentation editor now applies the style sheets of the selected presentation and a special designer style sheet that you can edit yourself: Help\Styles\TopicDesigner.css.  The designer sheet formats XML documentation markup such as <c> and <see> tags.  All of the style sheets are used in both the Topic Management and Topic Designer (shared content) dialogs.

image

Figure 5: API Topic Management Dialog - Xml Comments

The list of XML documentation sections is now generated automatically from DocProject's configuration file based on the type of topic that is selected.  Then the list is also merged with the actual elements found in the external XML documentation files, which allows users to add custom elements to external XML documentation files outside of the API Topic Management dialog and then use the dialog later to edit them.  There are other ways to manage custom elements, which I'll discuss later.

image

Figure 6: API Topic Management - Xml Comments Sections

The parameter that is selected in the image above was added by DocProject based on the reflection information that was provided by Sandcastle.  The method in the example only has one parameter, but if it had multiple parameters then there would be one param name="value" item in the list for each.

BTW, I just noticed a regression: the GenericClass should not show the back tick in its name - I'll fix that :)

You cannot delete delegate/method parameters or generic type parameters from the list.  You also cannot delete sections that may only appear once, such as summary or remarks.  But you can delete custom sections and sections that may appear multiple times, such as exception and permission, by right-mouse clicking and selecting Delete from the context menu.

To add an instance of a built-in section that may appear multiple times, you can select one from the drop-down list at the bottom.  The list displays only those items that are related to the API type of the selected topic.  You can also add custom sections by simply entering the name into the combo box and pressing Enter on the keyboard.

When a custom item is selected the attribute grid, which is displayed above the XML documentation editor, is completely editable, except for the Type column.  (Although I may change that - I haven't decided yet.)

You can create more built-in sections easily by registering them in DocProject's configuration file (commonly found at, C:\Program Files\Dave Sexton\DocProject\DaveSexton.DocProject.dll.config) under the sections element:

image

Figure 7: documentationTags element in the DocProject configuration file

XML documentation tag usage data has been retrieved from the following sources:

  1. Recommended Tags for Documentation Comments (C# Programming Guide)
    http://msdn2.microsoft.com/en-us/library/5ast78ax(VS.80).aspx
  2. The "Tag Matrix" that is present in Dynicity LLC's XML Documentation Comments Guide:
    http://www.dynicity.com/Products/XMLDocComments.aspx
    (Note: There are differences between the config data and the published matrix due to bug fixes and/or differences in versioning and tag/aspect applicability.)

Conclusion

There's much more, but I have to go catch a train now...

October 21, 2007

C# Multi-line XML Documentation Comments

I was just checking out the documentation on MSDN about XML comments and I discovered that C# supports multi-line comments for XML documentation as well:

/**
 * <summary>Hello World!</summary>
 */
public void HelloWorld() { }

Interesting, but I wouldn't recommend using multi-line comments for a few reasons:

  • They do not support Smart Comment Editing, where by Visual Studio inserts multiple comment lines and an empty summary tag automatically after starting the comment, which in this case is done by typing /** instead of ///.
  • Intellisense is limited:
    • top-level tags are listed but context is ignored (e.g., the para tag is never listed, even if the cursor is within a summary element).
    • Entering a tag name and pressing enter will not produce a closing tag automatically.
  • The rules for processing markup may be problematic since the compiler will look for a prefix pattern consisting of white-space, an asterisk and more white-space; however, a slight change in formatting in the prefix of one or more lines (e.g., less leading white-space) may cause the compiler to include one or more asterisks as part of the comments.
  • They are certainly not as common as triple-slash comments and will probably be confusing to most people (I know if I were to have seen this format used, and I already may have, I would have doubted whether it was even valid.)

I'm curious to know if anyone has found this feature to be more useful than triple-slash comments, and why :)

October 08, 2007

The Help! Project on CodePlex

I've just published the Help! project on CodePlex, which is intended to provide a fully managed compiled help system to replace Help 1.x (.chm).  Please contribute to this project by using the discussions area to post comments and ideas or join the development team.  If you want to join the team then please see the Welcome! thread for information.

For now I'll be using my blog to post news and information about this project.

October 08, 2007

DocProject 2008 Beta 2 Is Delayed

I've decided not to release DocProject 2008 Beta 2 with the 1.8.0 feature set.  Instead, I plan to reduce the number of conversion requirements and release Beta 2 with the next VS2005-compatible release, 1.9.0 RC.  Sorry for any inconvenience this may have caused.

October 04, 2007

DocProject 1.8.0 Release Candidate

DocProject 1.8.0 RC is now available for download on CodePlex.  It contains bug fixes as well as several new features:

  • Compatibility with the Sandcastle September CTP.
  • DocProject Properties context menu item for DocProjects and DocSites (on the project node in Solution Explorer) quickly opens a property page in Visual Studio Standard and higher.

  • DocSite Templates now support MSIE 6.0 (as well as IE 7.0, Firefox and Opera) and are themed, localizable, configurable and componentized.
  • DocSite Templates have a full-text search that supports complex query operations and uses an in-memory search index provider (based on .NET 2.0 Providers) that can be configured on the server, even after deployment.
  • DocSite Templates have a full-text index browse page that makes finding topics very easy.
  • The selected project Configuration and Platform work in MSBuild, Team Build and the DocProject External UI, which now displays drop-down lists for changing these values before building (like in Visual Studio).
  • The DaveSexton.DocProject.targets file has been refactored to make it more flexible.
  • Build output and Sandcastle project items are now located in a root folder named, Help.
  • After a build, compiled help is automatically copied to the project's output directory (for example, Debug\bin); however, for the DocSite templates the target is Help\bin. The DocSite TOC and index files are now generated in the App_Data folder.
  • DocProject External UI now supports the Include Project Output Dialog, which is automatically displayed after a build unless it's disabled by the user.
  • DocProject External UI provides a tools options page for configuring build engine providers just like the Engines tools options page that's available in Visual Studio Standard and higher.
  • Start Menu shortcuts to the DocProject External UI and various resources, including the Source folder in DocProject's installation directory, are now installed.
  • User configuration data is now stored in the system's isolated storage so that one user's changes do not overwrite another's; e.g., the default build engine provider and Sandcastle presentation style.

You can read more about the DocSite features in my previous blog post: DocSites 1.8.0 Preview

September 13, 2007

DocSites 1.8.0 Preview

In the upcoming DocProject 1.8.0 release candidate I've made some changes and added several substantial features to the DocSite templates that I'd like to share with you in this blog post. Changes to the DocSite templates include:

DocSite in Solution Explorer
  • ASP.NET Theme presentation:
    • Ships with one default theme named, "BasicBlue", which has the same appearance as the current DocSite templates.
    • Subtle improvements to appearance, mostly for Firefox and Opera.
    • Retained div-only + CSS layout with a sticky footer; i.e., no tables.
    • Includes some useful .SKIN files.
    • Full support for IE6 as well as continued support for IE7, Firefox and Opera.
  • Componentized Master page, with user controls for the header, footer, breadcrumbs, sidebar, TOC and index.
  • ASP.NET Localization with resource-based text in all pages and controls.
  • Customizable and highly extensible full-text search built from the ground up:
    • Designed for general-purpose indexing but customized specifically for indexing HTML help topics.
    • Has a default in-memory search index provider (I'll also build an SQL Server search provider for a subsequent release).
    • Complex query support with operators such as AND, OR, - (not) and grouping with parenthesis.
    • Customizable, factor-based weight and rank calculations.
    • URL query string supports bookmarking searches.
  • Keyword browse page for live browsing of the search index (also supports a URL query string for bookmarking).
  • Administration page for configuring DocSite options, search factors and for viewing simple search statistics.
  • New Project Wizard – Create DocSite Credentials step allows a user to set the administrative DocSite credentials when a DocSite is first created.
  • The DocSite templates reference a new class library that encapsulates common DocSite functions.

Breakdown of Features

The rest of this article will provide more information on some of the features listed above. If you have any questions or comments please let me know!

New Project Wizard – Create DocSite Credentials

The first change that you'll notice to the DocSite templates is a new wizard step:

New Project Wizard - Create DocSite Credentials
Figure 1: New Project Wizard - Create DocSite Credentials

This step allows you to enter an administrative user name and password for accessing the DocSite administration page (more on that later). The credentials are stored in the web.config file in the standard ASP.NET forms authentication section.

For custom DocSite templates authors can easily enable this feature by adding a single user to the authentication section with "$admin$" as the name (the password will be ignored). The mode of encryption that is chosen for the authentication section (e.g., Clear, SHA1, MD5) is reflected in the dialog's help paragraph automatically, and will also be applied to the user's password before it's stored in the web.config file.

Themes and Localization

The DocSite templates now use a default theme named, BasicBlue. Its appearance is almost identical to the appearance of the current DocSite templates, but now it has full support for IE6 as well as a sticky footer for all .aspx pages that do not use an IFRAME to display page content.

Supporting IE6 without a table layout was difficult, but I succeeded with the help of CSS expressions, which are specific to IE (but that means they're supported by IE7 too). The hard part was trying to support IE7's more standardized rendering engine without breaking IE6, and then supporting IE6 without IE7 falling back to things like CSS expressions. But after some flipping out and loss of hair, I finally got it working :)

Customizing the appearance of DocSites should be much easier now that it's fully themed. And with Sandcastle's high level of flexibility, anyone with some previous experience working with XSL and CSS should be able to come up with a custom Sandcastle presentation style and DocSite theme that work together for a completely customized appearance without affecting the DocSite's functionality, or even better, to improve on it. BTW, I'm certainly interested in hearing new ideas for themes if you have any.

Text that appears in all of the DocSite pages and user controls have been extracted into ASP.NET global and local resource files for easy localization.

Indexed Search and Browse

Finding help content in DocSites will be much easier now with the addition of the new search and browse features.

Note: The screenshots depict the Admin link as a different color than other hyperlinks, but that's simply because I clicked it.  What you see is the "followed" hyperlink color.  Before Admin is clicked it's the same color as the other links in the header: blue.

Search Page

DocSite Search Page
Figure 2: DocSite Search Page

The abstract, locale, namespace and library information are all taken from each topic's MSHelp data found in an XML data island generated by Sandcastle for Help 2.x. Although, due to a possible limitation in the next Sandcastle release, I might have to remove this feature temporarily until I can create a custom solution for a subsequent release.

DocSite search supports complex queries with operators that are defined by the search provider itself. By default, your DocSite will support the Boolean AND and OR operators, - (the negation operator) and parenthesized grouping:

DocSite Search Help
Figure 3: DocSite Search Help

The search help page is accessible by simply leaving the search box empty and clicking the search button (the one with the large magnifying glass).

NOTE: In the future I plan on adding an advanced search page that contains HTML topic-specific search functions, such as searching by the culture in which a topic was written, the topic's title, etc.

Browse Index Page

The button with the tiny magnifying glass, next to the search button, is the browse button. Clicking this button will open the browse page using the keywords entered into the search box, although all operators and grouping will be ignored. Clicking the button with an empty search box simply opens the browse page without any selected keywords.

Browse Index Page
Figure 4: Browse Index Page

The browse page allows you to drill down through a query by removing or adding one keyword at a time until you find a result set that contains the topic in which you're interested. The browse page is also useful for administrative purposes as it displays some statistical information about individual keywords as well.

The letter bar at the top of the page provides a list of keywords that start with the selected letter. Clicking a letter will display the keyword list. Then, clicking a keyword will add it to the filter and the new filtered results will be shown. You can continue to browse the index by the first-letter of keywords or view and modify the active filter at any time. The letter bar itself is encapsulated in a user control and is completely customizable using the administration page (discussed later).

Toggle between Search and Browse

The button next to the search query on the search results page and the button next to the keywords on the browse page allow you to easily toggle between these two distinct modes of locating help topics so that you can choose the one that is better suited for your particular query.

The Index

Both the search and browse features work with a full-text index of the HTML help topics generated by Sandcastle. The index is stored in-memory and generated on-the-fly by the website itself, not DocProject, which means that you don't have to rebuild the project to rebuild the index. It also means that you can tweak factors for weight and rank calculations and then rebuild the index on-demand through the web interface, allowing fine-tuning of the search results for common keywords that you might expect your end-users to search against. You can manage the search index and factors through the administration page, which I'll discuss later.

Your DocSite will automatically rebuild the in-memory index after the application has started upon the first request for the search page or a particular keyword on the browse page. In the future I plan on adding an SQL Server search provider that you can use instead of the default in-memory provider so that the index does not have to be rebuilt each time the ASP.NET worker process is recycled or when changes to the application are published that will cause it to restart (like certain modifications to the web.config file).

Site Administration Page

The DocSite administration page provides a set of configurable options with application-scope.

DocSite Administration Page
Figure 5: DocSite Administration Page

To reach this page simply click the Admin link in the top-right corner of your DocSite. If you haven't logged in then you will be brought to the login page first. Enter the credentials that you configured when the project was first created and then click the Login button to continue to the DocSite Administration page.

Settings

Settings are stored in the DocSite.config file found at the root of the project. Changes made directly to this file will not take effect until the application is restarted; therefore, I highly recommend using the web interface to configure settings whenever possible so that you do not have to restart the application (and lose the entire in-memory search index).

There are two main categories in the administration page: General and Search. Changes to any options in either category are applied immediately.

The Create Index link allows you to regenerate the index on-demand. This is useful for ensuring that the index is generated immediately after the DocSite has been published and for updating the index after modifying keyword, weight or rank settings.

Client Settings

Letter bar

Configure the characters that will appear in the letter bar at the top of the browse index page. Add characters from your native language or remove existing characters without having to modify any code.

Sidebar size persisted

Controls whether a cookie will be used on the client to remember the last position of the sidebar's handle. This settings is enabled by default.

The Search category has a few subcategories for viewing statistics and managing the DocSite Search and Browse Index pages.

Index Statistics

This category provides read-only information:

Provider name

Name of the current search index provider. DocSiteMemorySearchProvider is the only implementation that will ship in DocProject 1.8.0; however, expect SqlSearchProvider in a subsequent release.

Last creation date

The last date and time that the index was created.

# keywords

The total number of distinct keywords found among all documents that were indexed.

# documents

The total number of HTML help topic files that were indexed.

Settings

Provides general index-related settings:

Root search path

Virtual path of the root directory in which the provider will begin the search, recursively. I highly recommend that you do not change this setting.

Public search enabled

Indicates whether unauthenticated users can use the DocSite search feature. Note that if you're logged in after disabling this setting you will still see the search box since it's always available to authenticated users. This setting is enabled by default.

Public browse enabled

Same as the option above but for the DocSite browse index page instead of the search page. This setting is enabled by default.

Keyword Settings

Settings related to indexed keywords and keywords in search queries.

Minimum keyword length

Defines the minimum number of characters that a word must contain in order to be included in the index. This setting is also used to ignore words in queries that have fewer characters than the specified value. The default value is 2.

Excluded keywords

Comma-separated list of keywords that must be excluded from the index and queries. You can add or remove keywords as you see fit.

Hot title keywords

Comma-separated list of keywords that have special semantics in titles of HTML help topics and are used to add additional value to matches in query results through the Title Rank Factors values (below).

You can customize this list if you want to improve search results with queries that contain full or partial matches to these particular words when found in titles.

Weight Factors

Weight factors are integers that affect the weight of indexed keywords. Note that weight is calculated while the index is being created and rank is calculated when a keyword matches a search query. The values in this section do not affect rank directly; however, the weight of a keyword does add additional rank to matches.

Early position keyword

Factor used to calculate the weight of a keyword depending upon its location in the document being indexed. Typically, the earlier the position the higher the weight.

The formula used to calculate weight is:

keyword position / source length * -{value} + {value}

Where {value} is the value of this factor (10 by default).

Title keyword

Factor used to calculate additional weight for keywords found in the titles of indexed documents. The base weight of title keywords is the value of the Early factor (above), but calculated relative to the title and not to the entire document. The value of this factor is then multiplied by the result. The default value is 1.2, which gives 20% more weight to keywords found in titles.

Title Rank Factors

Additional rank values for keyword matches that are found in titles.

Exact title hot keyword match

Additional rank for exact keyword matches found in titles and the Hot title keywords list (above). The default value is 250.

Partial title hot keyword match

Additional rank for partial keyword matches found in titles and exact matches found in the Hot title keywords list (above). The default value is 100.

Exact title keyword match

Additional rank for exact keyword matches found in titles. The default value is 50.

Partial title keyword match

Additional rank for partial keyword matches found in titles. The default value is 20.

Conclusion

In this blog post I described new features to come for the DocSite templates in DocProject 1.8.0 RC. These new features will surely be appreciated by your end-users since they make it much easier than before to locate and browse help topics. As a site administrator you now have a much easier way to configure some of the DocSite features and will have an easier time modifying the appearance and localizing text to meet your particular needs and the needs of your clients.