using Application.ActiveDocument safely

In Word 2007 I would always check whether there were any document in my current application before I call Application.ActiveDocument. This is because if you call this method and there are no documents it would throw an exception.

Now developing in Word 2010, it only gets more complicated. There is now a ActiveProtectedViewWindow as well and this document cannot be retrieved in the applications document list.

So I created a helper method to get the current active document. Here's the code

Issues with AltChunk

I managed to find the underlying problem to a rather annoying bug using AltChunk. I've generated thousands of reports using OpenXml but this one word document would always break xml markup of the document when inserting it using AltChunk.

After stripping the document to only contain the parts that broke it I managed to find out that the "DocumentSettingsPart" contained some elements called SmartTagType. I've never seen these before and not sure what they are used for but the moment I removed them from the document my AltChunk insertion started to work so I now remove them from all my documents before I insert using AltChunk.

I wonder if this is a known bug - will ask on the forum.

Here's some code:

// Get a list of smart tags in the document settings part and remove these
List<SmartTagType> smartTags = mainDocumentPart.DocumentSettingsPart.Settings.Descendants<SmartTagType>();

// Loop backwards otherwise the elements orders change
for (int i = smartTags.Count - 1; i >= 0; i--)
{
  smartTags[i].Remove();
}

Maintain original image size when moving images in open xml

When inserting images into content control boxes using openxml, the image dimensions will not change to those of the image, you will need to manually do this.

Which objects?

There are two places where the image size needs to be set, inside the DocumentFormat.OpenXml.Drawing.Pictures.Picture object and the DocumentFormat.OpenXml.Drawing.Wordprocessing.Inline object. Both of these objects contain an DocumentFormat.OpenXml.Drawing.Wordprocessing.Extent object which contains the Cx and Cy values for the image.

What size?

Now the next problem is knowing what the Cx and Cy values should be set to. Use the System.Drawing.Image object to get the Width and Height properties from your image. However, word image dimensions are not stored in pixels, they are stored in points.

Converting pixels to points can be very complicated when you're working with text but luckily with images there is a constant value we can use to calculate the points from a pixel value, which is 9525.

Code example please?

Trouble with OpenXML and MemoryStreams?

I've been struggling for a while with changing my project to use Memory Streams instead of physical files. There were a few things I was doing wrong which you probably wouldn't pick up if you started off using physical files but because I was now refactoring I was missing a few pointers.

A) MainDocumentPart.Document.Save()

When working with the physical file, there is no need to save but when working with a memory stream you need to call this method just before you close your word document.

B) MemoryStream.WriteTo(fileStream)

Because I was working from a document that already existed I would open the document and not see any changes. This was because I was not saving the document back to the file. Very rookie, I know, but it happened and maybe this helps someone out there. More importantly see next point:

C) When to save your memory stream back to the file!!!!

I was writing my memory stream back to the file while the file was still open. This did not throw any exceptions and it still does not make sense to me why this is a problem but it is! Make sure that you always save the memory stream to a file after you have closed the word document but when your memory stream is still open. This caused me hours of pain :( If anybody knows why this happens please let me know.

Here is an example of what your code should look like

Moving images in word headers using OpenXML

From what I discovered moving images from one header in a document to another seems to be a huge problem for most people so I decided to try do this myself. I'm not 100% sure what other issues it might cause but I when I run the "Validate" command in the OpenXML Productivity Tools I get no errors and the images I'm moving are displaying correctly so for now I'm giving it the go ahead :)


Background


To move data from one document to another you can use altChunk but this does not copy headers and footers. If this doesn't phase you - you can read more about altChunk on Brian Jones blog: entry: http://blogs.msdn.com/b/brian_jones/archive/2008/12/08/the-easy-way-to-assemble-multiple-word-documents.aspx.


Brian Jones has tons of examples of how to manipulate documents. He also has some nice information about some open source code called "DocumentBuilder" which I stumbled upon but when I run the "Validate" command using the OpenXML Productivity Tools I get an error for each document I tried to merge so I decided this was not a good option for me. If you are interested though check it out here:  http://www.pubsub.com/events/5e5f8d3fd2346a54739b8b6bb0438b82


Another way to move elements from one part of a document to another is to actually use the "FeedData" method and pass in the other element by getting its Stream. This however doesn't work when moving images in headers between documents.

Looking Deeper

Create a blank document and insert any image into it's header. Use the OpenXML Productivity Tool to open the document and expand it's /word/document.xml node. You will see that there are few header xml nodes inside the document xml node. Thats because there are 3 types of headers inside a document or none at all. The three types distuinguish between First, Default and Even.


If you go to expand "w:document", "w:body" you will see "w:sectPr". If you reflect this node you will see the references to your headers. In my document my "Default" header has been referenced using id "rId8". If you reflect your "/word/document.xml node you should see which Header has been assigned "rId8". This helped me to find out which HeaderPart I need to retrieve to find the elements I'm looking for.


Start Coding


A HeaderPart can contain a few things but if I just have a plain header with an image it should contain a "/word/media/image1.png" element as well as a "w:hdr" element. What I did was create a new Header by passing the old headers OuterXML into the constructor and manually copy paste the Images seperately and this worked. Here is some example code:


reportImagePart.FeedData(headerImagePart.GetStream());

private static void ReplaceImages(HeaderPart headerPart, HeaderPart reportHeaderPart) 
{ 
  foreach (IdPartPair partPair in headerPart.Parts)   
  {
    OpenXmlPart openXmlPart = partPair.OpenXmlPart;   
    Type type = openXmlPart.GetType();   
    if (type.Name == "ImagePart")     
    {
      ImagePart headerImagePart = (ImagePart)openXmlPart;     
      string idImage = headerPart.GetIdOfPart(openXmlPart);     
      ImagePart reportImagePart = reportPart.AddNewPart<ImagePart>("image/png", idImage);
      Header headerNew = new Header(headerPart.Header.OuterXml);
    }
  }
}

This could probably be coded better but I'm still in the POC process. If anyone is having any issues with this strategy please let me know!!!

Metadata Property Mappings

In order for me to make full use the FullTextSqlQuery SharePoint object which I spoke about in my previous blog here I needed to understand how metadata property mapping worked. This is my "lamens term" explanation: There are "Crawled Properties" which as a SQL developer you could think of as an indexed field. Then you get a "Managed Property" which you can map to multiple "Crawled Properties". The "Managed Property" is the field that will be available to use. You could think of this property as an interface to your indexed field(s).

So hopefully that doesn't make things more complicated, moving forward this is how you would go about creating Metadata Property Mapping:

To navigate to the Property Mapping Page go to Shared Services Provider. On the home page click on the "Search settings" option under "Search". The navigation menu should have changed now. Click on the "Metadata properties" item in the quick launch menu under "Queries and Results". On this screen you can see all the managed properties and their crawled property mappings. From this page you are able to see all meta data properties. 

If you navigate to the "Title" propery and click on it you will see the following screen:

Let me explain what all these properties mean:

A. The type of information in this property. The available types are (Text, Integer, Decimal, Date and Time, Yes/No). Note that you can only link crawled properties that are the same type as your managed property.

B. The number of items found with this property. This is a very useful indication of whether your managed property will actually return any results. If this value is 0 is means one of two things.
1. Your mapped crawled properties do not contain any values
2. You haven't run a crawl yet. After you have created a managed property always run a crawl in order for SharePoint to index it.

C. Option "Include values from all crawled properties mapped" means that all mapped crawled properties will be returned in a deliminated fashion.
Option "Include values from a single crawled property based on the order specified" means that it will only return the first value that is not null depending on their order in the list box.

D. Crawled properties mapping to this managed property.
On the title example you will see that the first mapped property is "Mail:5(Text)". The keyword "Mail:" means that this crawled property will be retrieved out of a mail object. This full name is not very descriptive to the user but you can just google a field or test it out to find out what value it contains.

If you click on the "Add Mapping" Button you will be able to search for your desired property. If you are looking for a column inside a list or library you will find them under the "SharePoint" category. All custom column's crawled property names will begin with "ows_" and spaces will be converted to "_x0020_". Note: if you cannot find the property you are looking for check whether they are of the same type as your managed property. This has got me before ;)

E. Allow this property to be used in scopes.
This boolean value can be misleading in it's description. If this checkbox is selected it means that you will be able to create a rule in your scope that can use this property. If this property is unchecked you can still use scopes in conjunction with this field in your searches.

So one last thing to remember is ALWAYS RUN A CRAWL after any updates!!!

SQL Query in SharePoint

The "FullTextSqlQuery" object's constructor requires an object that has context but don't be fooled. This context will not be the context in which your query will search on. Scopes are used for that are I will be explaining scopes a little later on.

Query Text

The "FullTextSqlQuery" object has a "QueryText" property which takes a query very similar to a generic SQL query. Here are a list of minor differences for those who are interested:

• There is only one table to search on - "Scope" and this table is suffixed by parenthesis.
• You cannot search on all columns using *. You have to specify the column names you require.
• There are no aggregate functions eg. 'Count'
• I found that "LIKE" doesn't work. I have found examples that use "LIKE" and state that it works. I myself never got the results I was expecting so I use "CONTAINS(Title, 'SharePoint').
• The comparable commands are Equals "=", Not Equals "!=", Contains "CONTAINS(ColumnName, 'Value')" and FREETEXT(*, 'value') which allows you to search on all columns. NOTE: You cannot put spaces in any of these value objects. Replace all spaces with the '+' character to search multiple words. And use the '*' character in the values as a wildcard - WILDCARDS ONLY WORK IN SP 2010!.

So here is an example of what your QueryText could look like:

SELECT Title, Description, Path FROM SCOPE() WHERE CONTAINS(Title, 'SharePoint') AND IsDocument = 1 ORDER BY Title

Query Properties

There are some properties on the "FullTextSqlQuery" object that you will need to set.

• ResultTypes. This is an enum of type Microsoft.Office.Server.Search.Query.ResultType. I've only ever used the "RelevantResults" value. I've tested the rest but I don't get any results using them so not sure what they are for.
• TrimDuplicates. As the name states.
• StartRow. Used for paging. The index of the first item to return. NOTE: If you're using the API this index starts at 0 but if you are using the webservice to call a Query the index starts at 1.
• RowLimit. Used for paging. The amount of results the query must return.

There are some more properties but I'm not sure exactly what they are used for yet.

Columns

If you want to know what columns you can select on you can see this in your Shared Services Provider. On the home page click on the "Search settings" option under "Search". The navigation menu should have changed now. Click on the "Metadata properties" item in the quick launch menu under "Queries and Results". On this screen you can see all the managed properties and their crawled property mappings. Check about my other post "Metadata Property Mapping" for more information.

Scopes

Scopes allow us to search in specific areas of sharepoint. I haven't worked extensively with this yet but from what I understand at the moment you can only create a scope inside a web application and a site collection. This means that if you have multiple web applications and multiple site collections you can make your search specific to one of multiple of these. If you don't specify a scope the search will return results from all web application and all site collections. This optomises your search greatly as the scopes index all your results. NOTE: In your for your scope to be visible from your "FullTextSqlQuery" it needs to be Shared.

Indexing

All "Managed properties" and "Scopes" are indexed by SharePoint. When a new object is added to SharePoint is will not be indexed yet and will not be returned by your search query until this has happened. A crawl needs to be started in order for any new objects or object changes to appear in your results. If you are busy testing your search you can manually start a full crawl inside your Shared Services Site. On the home page click on the "Search settings" option under "Search". The navigation menu should have changed now. Click on the "Content sources" item in the quick launch menu under "Crawling". You can start a full crawl on your content source on this page. Once the crawl is complete run your search to see if your results are being returned. If you want to deploy a custom search feature you will need to setup an incremental crawl for this content source so that it executes every so often.

Results

The results of a "FullTextSqlQuery" are returned in the form of a "ResultTableCollection" from the "Execute" method. In order to get your results from this collection you need to the index the collection with your ResultType like so "fullTextSqlQuery.Execute()[ResultType.RelevantResults];".
The ResultTable is not very straight forward and I struggled to get my results out. First you need to call the Read() function inside a while loop and then use a for loop to get the index of each result item. You then use two methods on the table: "GetName()" and "GetValue()". Each of these methods take an index parameter. I then need to compare the name to the name im looking for and set the value for the object. Here's a code sample:

while (resultTable.Read())
{
  for (int i = 0; i < resultTable.FieldCount; i++)
  {
      string name = resultTable.GetName(i);
      object objValue = resultTable.GetValue(i);
      string value = string.Empty;
      if (objValue != null)
      {
        value = objValue.ToString();
      }

      if (name == SearchProperties.Title.ToString().ToUpper(CultureInfo.CurrentCulture))
      {
        sr.Title = value;
      }                       
   }
}

Bugs
Yup, like most complex software out there, even Microsoft software, there are bugs. So far I have picked up two.

1. Sort by Author does not return desired results. The Author managed property is linked to a Mail and Office category field. For some reason if you sort by this property it adds a null row after every result so your count will double. As far as I'm aware this is a bug in 2007 and 2010.

2. ModifedBy and CreatedBy are empty. Although these fields exist inside the SPItem, you will not be able to retrieve them using the FullTextSqlQuery object. It seems that these fields are not indexable and they will always return null.

Resources
1. http://community.bamboosolutions.com/blogs/bambooteamblog/archive/2009/04/24/wss-custom-search.aspx
2. http://blogs.msdn.com/b/varun_malhotra/archive/2008/08/16/moss-search-with-order-by-clause-doesn-t-return-all-results.aspx