Adam Toth’s Blog
Mar 30 2009
I’ve just released a beta of xPollinate, my Windows Live Writer Cross Post Plugin.
You can download the installer at the CodePlex site, and I’ll shortly post it on the Live Writer plugin gallery.
As an example, I’ve used it to cross-post this entry to my old Blogger account, as well as via Ping.FM to my Twitter feed (status method) and as a full blog post to my Tumblog.
UPDATE 3/31/09: Plugin is now available on the Live Writer Gallery here.
Mar 19 2009
I’ve seen several examples of creating document converters in SharePoint that convert documents to PDF format, using tools like Aspose and other print-based solutions.
I recently did a proof of concept for a customer that used a different product, Outside In PDF Export from Oracle. This is an interpretive converter that can take a slew of file formats (including MS Office) and automatically convert them into a PDF file. If you aren’t familiar with PDF conversion, there are two basic kinds:
Outside In, and in particular the PDF Export SDK, is an EXE and set of DLLs that contains logic to covert over 400 different file types to PDF. In combination with Transformation Server (a web services wrapper to PDF Export), you can create SharePoint Document Converters that will convert your documents to PDF format.
For the proof of concept, I followed these steps to get it working
You can download both products here:
http://www.oracle.com/technology/products/content-management/oit/oit_dl_otn.html
You’ll have to register with an Oracle account to get it, but it is a freely available trial version, and does not appear to have any trial expiration limitations.
You will also need SrvAny.exe to run Transformation Server as a Windows Service. I found this site that had a download:
http://www.tacktech.com/display.cfm?ttid=197
Extract the contents of the Transformation Server zip you downloaded to a folder on your server that will do the document conversions (e.g. “C:\PDFConverter”). There will be a bunch of dlls, some wsdl files (we’ll use those later), some EXE’s (which are run as a service), and some XML config files.
Transformation Server doesn’t really know anything about PDF generation, it is just a web service wrapper, so we’ll need to place the PDF Export library into our folder so it can use it. Extract the contents of PDF Export 8.3.0 to a separate folder on your machine. Grab every DLL in the root folder, as well as exporter.exe, px.cfg, and cmmap000.bin and copy those files into “C:\PDFConverter”.
Open the file called server_startup.xml in “C:\PDFConverter”. This configures the service to listen on a specific port and hostname/ip address. I went ahead and changed the port, but you can leave it as is if you wish, just make note of the port for later:
<!--Connections are the TCP/IP socket connections that the server accepts from clients.-->
<connectionsInfo xsi:type="tss:ConnectionsInfo">
<!--The serverName is the host name, or dotted IP address that the server will use when establishing its presence on the network.-->
<serverName xsi:type="xsd:string">localhost</serverName>
<!--The port is the TCP/IP port that the server will establish a listener on for accepting connections.-->
<port xsi:type="xsd:unsignedInt">9000</port>
<!--The server will disconnect idle connections that have remained idle for too long. The activityTimeout sets the period of time a connection must be idle before it will be disconnected by the server.-->
<activityTimeoutSecs xsi:type="xsd:unsignedInt">1800</activityTimeoutSecs>
</connectionsInfo>
Go ahead and double-click tsmanager.exe to make sure it is working. It should open a console window and show you what host/port it is listening on.
Press Ctrl + C to quit.
Now we need to run this program as a service, so that it is always available, even when nobody is logged on to the server. Place instsrv.exe and srvany.exe in your C:\PDFConverter folder. Run the following command prompt:
C:\PDFConverter>instsrv.exe "Transformation Server" c:\PDFConverter\srvany.exe
The service was successfuly added! Make sure that you go into the Control Panel
and use the Services applet to change the Account Name and Password that this
newly installed service will use for its Security Context.
Open regedit, and browse to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Transformation Server.
Create a new key called Parameters, and add a new string value inside this key called Application, with a value of C:\PDFConverter\tsmanager.exe.
Go to the services console and start the Transformation Server service. Optionally, you can also go to the Properties for this service account, and change the Identity that this service runs under.
Create a new .NET C# console application project and solution from within Visual Studio 2008, called PDFConverter.
Right-click the project in Solution Explorer and choose Add Service Reference…
Since the wsdl files for Transformation Server are not compliant with WCF services, choose the Advanced… button on the Add Service Reference dialog, and then choose the Add Web Reference… button to add a .NET 2.0 style web service reference. In the Add Web Reference dialog, enter the path to the wsdl file: C:\PDFConverter\transform_net_2005.wsdl. Hit Go.
Once your web reference has resolved, enter TransformationServer in the Web reference name box and click the Add Reference button.
Open your Properties/Settings.settings file. There will be one setting in the file for the web service URL. Change this to the right hostname/port in your server_settings.xml for Transformation Server. Make sure to include a trailing “/transform” on the end, e.g.:
http://localhost:9000/transform
A document converter is just a console EXE that is called by SharePoint with command line arguments, so we’ll setup our console app to handle the arguments. In Program.cs, before static void Main, add the following code (taken from the WSS SDK here):
#region Argument Handling Data
private enum ArgumentCode
{
Help,
InputFile,
OutputFile,
LogFile,
ConfigFile
}
private struct ArgumentPair
{
public ArgumentPair(ArgumentCode c, string s)
{
this.argType = c;
this.argFlagString = s;
}
public ArgumentCode argType;
public string argFlagString;
}
private static ArgumentPair[] PossibleArguments = new ArgumentPair[] {
new ArgumentPair(ArgumentCode.Help, "-?"),
new ArgumentPair(ArgumentCode.Help, "-help"),
new ArgumentPair(ArgumentCode.InputFile, "-in"),
new ArgumentPair(ArgumentCode.OutputFile, "-out"),
new ArgumentPair(ArgumentCode.LogFile, "-log"),
new ArgumentPair(ArgumentCode.ConfigFile, "-config"),
};
private static Dictionary<ArgumentCode, string> actualArguments = new Dictionary<ArgumentCode, string>();
#endregion
/// <summary>
/// Prints out the usage help to the console.
/// </summary>
static void PrintUsage()
{
Console.WriteLine("Usage: {0} -in <inputfilename> -out <outputfilename> -config <configfilename> [-log <logfilename>]",
Assembly.GetExecutingAssembly().ManifestModule.Name);
}
Add the following code inside static void Main:
#region Argument Handling Code
// Read arguments.
for (int i = 0; i < args.Length; i++)
{
bool argumentValid = false;
foreach (ArgumentPair ap in PossibleArguments)
{
if (String.Compare(ap.argFlagString, args[i], StringComparison.OrdinalIgnoreCase) == 0)
{
switch (ap.argType)
{
case ArgumentCode.Help:
PrintUsage();
return;
case ArgumentCode.InputFile:
case ArgumentCode.OutputFile:
case ArgumentCode.ConfigFile:
case ArgumentCode.LogFile:
if (i + 1 < args.Length)
{
actualArguments[ap.argType] = args[i + 1];
i++;
argumentValid = true;
}
break;
}
break;
}
}
if (!argumentValid)
{
Console.WriteLine("unknown argument {0}", args[i]);
PrintUsage();
return;
}
}
// Validate arguments.
if (!actualArguments.ContainsKey(ArgumentCode.InputFile) ||
String.IsNullOrEmpty(actualArguments[ArgumentCode.InputFile]) ||
!actualArguments.ContainsKey(ArgumentCode.OutputFile) ||
String.IsNullOrEmpty(actualArguments[ArgumentCode.OutputFile]) ||
!actualArguments.ContainsKey(ArgumentCode.ConfigFile) ||
String.IsNullOrEmpty(actualArguments[ArgumentCode.ConfigFile]))
{
Console.WriteLine("required argument missing");
PrintUsage();
return;
}
#endregion
Now that the command line arguments can be parsed, it is time to use the web service and do the document conversion. The Transformation Server download contained some code samples for C#, and I was able to basically use as-is, only modifying the input and output file paths. After the argument handling code in static void Main, add the following code:
TransformationServer.transform t = new TransformationServer.transform();
TransformationServer.IOSpec source = new TransformationServer.IOSpec();
source.spec = new TransformationServer.stringData();
source.spec.base64 = false;
source.spec.charset = TransformationServer.CharacterSetEnum.windows1250;
source.spec.str = actualArguments[ArgumentCode.InputFile];
source.specType = "path";
TransformationServer.IOSpec sink = new TransformationServer.IOSpec();
sink.spec = new TransformationServer.stringData();
sink.spec.base64 = false;
sink.spec.charset = TransformationServer.CharacterSetEnum.windows1250;
sink.spec.str = actualArguments[ArgumentCode.OutputFile];
sink.specType = "path";
string outputFormat = "pdf";
string optionSet = "";
TransformationServer.Option[] options = new TransformationServer.Option[0];
TransformationServer.stringData resultMsg;
TransformationServer.IOSpec[] resultDocs;
UInt32 result = t.Transform(source, sink, outputFormat, optionSet, options, out resultMsg, out resultDocs);
Console.WriteLine("The result is {0} {1}", result, resultMsg.str);
foreach (TransformationServer.IOSpec ios in resultDocs)
{
ASCIIEncoding encoding = new ASCIIEncoding();
string constructedString = encoding.GetString(Convert.FromBase64String(ios.spec.str));
Console.WriteLine(" {0}", constructedString);
}
Most of this code is cut-and-paste from the sample, and we are just swapping the input file path (source.spec.str) and output file path (sink.spec.str) with the arguments from the command line.
To install the converter, you need to create a feature file, install the feature, and place the converter EXE in the proper directory.
Create a feature.xml file and add the following:
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"
Id="{9E2A5231-0D81-4de8-8504-36A61026680E}"
Title="PDF Converter"
Description="PDF Converter."
Scope="WebApplication">
<ElementManifests>
<ElementManifest Location="Elements.xml"/>
<ElementFile Location="PDFConverter.exe"/>
</ElementManifests>
</Feature>
Create an Elements.xml file, and add the following code. You can create as many document converter nodes as you need for each file type you want to convert (just make sure each has a unique guid).
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<DocumentConverter Id="{513130EB-A1BC-46d3-919C-1E00B6BCA742}"
Name="DOCX to PDF Converter"
App="PDFConverter.exe"
From="docx"
To="pdf"
/>
<DocumentConverter Id="{E7374437-157E-4fb4-A4E9-A19026BAE17A}"
Name="VSD to PDF Converter"
App="PDFConverter.exe"
From="vsd"
To="pdf"
/>
</Elements>
Place both these files in a folder called PDFConverter in your …/12/TEMPLATE/Features folder, and use stsadm to activate the feature on a particular web application.
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>stsadm -o installfeature -name PDFConverter Operation completed successfully. C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>stsadm -o activatefeature -name PDFConverter -url http://mossdev:8080 Operation completed successfully.
After activating the feature, go to Central Administration, Applications tab, and click the Document Conversions link. Ensure that your web application has document conversions turned on. You should see your document converters in the list at the bottom of the screen:
Since you are using a Web Service, a proxy class needs to be generated dynamically, and so you’ll need to give your SharePoint Document Conversion User Account rights to create this in C:\Windows\Temp (or whatever your system temp directory is). Grant List Folder Contents and Read permissions to the Document Conversion account (usually a local machine account that starts with “HVU_”) on your Temp folder.
The Document Conversion service stores temporary documents in the following folder: C:\Program Files\Microsoft Office Servers\12.0\BIN\HtmlTrLauncher. By default, this folder only grants the Document Conversion User account access. Since the Transformation Server windows service is running under its own account, you need to give this account access to the folder. Grant the account that your Transformation Server windows service is running under Read and Write access to this folder.
The last step involves copying over your compiled console EXE and exe.config file into the following SharePoint folder (the actual path may vary depending on your setup, but you should be able to find the TransformApps folder):
C:\Program Files\Microsoft Office Servers\12.0\TransformApps
Navigate to your SharePoint site. Upload a document into your document library (make sure it is a file extension that you’ve configured in your Elements.xml file). Open the drop down menu and choose Convert Document. Your document converter should appear in the list. Run the conversion. After around a minute if you refresh your document library and all went well, you should see your PDF file in the library.
Here is a picture of a generated Visio document in PDF format, not too bad:
Mar 17 2009
I am just finishing up the touches on my first Windows Live Writer plugin, which I’m calling xPollinate. I’ve wanted to cross post to multiple blogs as well as Ping.FM from Windows Live Writer, and originally I found this plugin by Daniel Cazzulino that was not perfect, but got the job done.
I used this plugin as a basis and then added a bunch of bells and whistles that I wanted. Following are some feature enhancements that I’ve added:
Here are some screen shots:
The Blogs Tab:
This enables you to select multiple blogs to cross post to. You can optionally post a summary, and include header and footer text.
The Ping.FM tab:
This enables you to post your blog entry via Ping.FM to your configured services. You can post a full or summarized version to your Ping.FM blogs, and can post a notification to your Ping.FM status and microblog services. The format of the notification is customizable.
Left to Do
I’ll be releasing this as a free plugin to the community, and will release the source as well. The final few things I have left to do are:
Mar 10 2009
When I was a kid, I remember my doctor really taking care of me. There were just things that doctors did that were expected of them, such as:
After taking my son in to be looked at for Pink Eye, I was amazed at how much things have changed.
We didn’t clean his eyes before the visit, to make sure that the doctor saw the discharge. She looked at his eyes which were all goopy, and didn’t even clean them out when she was done.
I guess they didn’t have any eye drops on hand to offer the first dose onsite, but even if they had, they wouldn’t have done it anyway.
His sister’s eyes were also starting to get goopy, and it was a good bet at the time that she got what her brother had. Of course, the doctor couldn’t offer any suggestions for her, she needed a separate visit because “she was not in the system yet”. Along those lines, she told us that his sister couldn’t use his drops because she was under a year old, and they typically use an ointment instead of drops. I guess I just wondered why she couldn’t have prescribed both meds for him, and let us use the ointment on her, since she was clearly progressing along the same lines as her brother.
I don’t know what the reasons are, maybe it’s fear of lawsuits, maybe it’s financially motivated to increase office visits, maybe it’s just a shift in expectations that’s occurred as a result of limited time and budgets. In any case, it ain’t what it used to be.
Mar 06 2009
While working on a Timer Job project, I needed a flexible way to store one-time configuration settings. I was using a SharePoint Feature and FeatureReceiver to install and activate the Timer Job, so I decided to use the Feature.XML file as a repository for the initial configuration settings Job.
I used the Properties node of the feature.xml file to store the settings:
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"...>
<Properties xmlns="http://schemas.microsoft.com/sharepoint/">
<!-- The name of the timer job (will appear in Central Admin). -->
<Property Key="JobTitle" Value="Your Timer Job Name Here"/>
<!-- Connection String that the TimerJob will use -->
<Property Key="ConnString" Value="ConnectionString..."/>
<!-- The schedule to run the job in (24 hour format) -->
<Property Key="Schedule" Value="daily at 02:00:00"/>
</Properties>
</Feature>
Then, during the FeatureActivated event, I passed those values into the Properties bag on the Timer Job class:
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
// Get all the properties from the feature.xml file
string jobTitle = properties.Feature.Properties["JobTitle"].Value;
string connString = properties.Feature.Properties["ConnString"].Value;
string dailySchedule = properties.Feature.Properties["Schedule"].Value;
// Create the job.
CustomTimerJob customTimerJob = new CustomTimerJob((SPWebApplication)properties.Feature.Parent, jobTitle);
// Set the properties for the job to run properly
customTimerJob.Properties.Add("ConnString", connString);
customTimerJob.Properties.Add("Schedule", dailySchedule);
// Set the schedule
SPSchedule mainSchedule = SPSchedule.FromString(dailySchedule);
customTimerJob.Schedule = mainSchedule;
// Activate the schedule
customTimerJob.Update();
}
Then, in the Timer Job’s Execute method, grabbed those settings and used them:
public override void Execute(Guid targetInstanceId)
{
// Set up configuration values
string connString = this.Properties["ConnString"] as string;
...
}
I didn’t have a need to make run-time changes to the configuration settings, so this worked out really well. To change the defaults and apply new settings, the process was as simple as:
Mar 06 2009
I enjoy working with the ASP.NET Wizard control – it’s one of the more useful controls available, and there always seems to be a place for a wizard in the custom apps I’ve written. Over time I’ve developed a helper method that makes it easier to do non-linear jumps from step to step.
Often, in a NextButtonClick event handler for a Wizard, you need to figure out what the current step is that you are on, based on the WizardNavigationEventArgs e.CurrentStepIndex property. Here is a helper method to enable you to do that:
/// <summary>
/// Returns the index of a particular wizard step in the WizardSteps collection
/// </summary>
/// <param name="step">The step whose index number you want</param>
/// <returns></returns>
private int GetIndexFromStep(WizardStepBase step)
{
return step.Wizard.WizardSteps.IndexOf(step);
}
You can also package it up as an extension method (.NET 3.5) off of the Wizard or WizardStepBase object:
using System.Web.UI.WebControls;
/// <summary>
/// Extensions for ASP.NET Wizard Control Objects
/// </summary>
public static class WizardExtensions
{
/// <summary>
/// Gets the index of a particular wizard step.
/// </summary>
/// <param name="wizard">The current Wizard</param>
/// <param name="step">The step whose index number you want</param>
/// <returns></returns>
public static int GetIndexFromStep(this Wizard wizard, WizardStepBase step)
{
return wizard.WizardSteps.IndexOf(step);
}
/// <summary>
/// Gets the index of this wizard step.
/// </summary>
/// <param name="step">This step</param>
/// <returns></returns>
public static int GetIndex(this WizardStepBase step)
{
return step.Wizard.WizardSteps.IndexOf(step);
}
}
If you’ve given your wizard steps specific IDs, then you can use them strongly typed in a NextButtonClick event:
protected void Wizard1_NextButtonClick(object sender, WizardNavigationEventArgs e)
{
if (e.CurrentStepIndex == GetIndexFromStep(step1))
{
// We are on step #1
// Validate...
// Skip to step 3
Wizard1.ActiveStepIndex = GetIndexFromStep(step3);
}
}
Mar 05 2009
I’m working on a project that imports over 1 million users from an Oracle database used with SharePoint/Forms Authentication into the SharePoint user profile store. This is done as a custom SharePoint timer job that pulls the users from the DB and creates/updates User Profiles through the SharePoint API.
When running a job on a recordset of this size, there are several things to strive for:
Two ways you can achieve this:
Most code samples on the web that deal with programmatic creation of User Profiles will show code such as this:
if (profileManager.UserExists(accountName)
{
userProfile = profileManager.GetUserProfile(accountName)...
}
else
{
userProfile = profileManager.CreateUserProfile(accountName)...
}
On small recordsets, this is fine, but for large recordsets the UserExists method represents a bottleneck that can increase the duration that your process runs. In addition, in the code above, you will unknowingly call this method a second useless time, because the CreateUserProfile() method internally calls UserExists() as well.
There are two ways to avoid this method:
The UserProfileManager object is an IEnumerable that you can iterate over and access all the Profiles in SharePoint. Caching the IDs of these profiles up front enables you to index into a Dictionary object to see if your profile exists, rather than hitting SQL Server with UserExists(). The following code helped to reduce processing time significantly (you take a hit up front, but it’s far less than the delay imposed by UserExists over large recordsets):
Dictionary<string, Guid> cachedProfiles = new Dictionary<string, Guid>();
foreach(UserProfile profile in profileManager)
{
cachedProfiles.Add(profile.AccountName, profile.ID);
}
...
if(cachedProfiles.ContainsKey(accountName)
{
...
}
In addition, caching the Guid of the UserProfile lets you later use the overloaded method of GetUserProfile() that takes a Guid as a parameter, which seems to perform slightly better than the alternative that takes a string for AccountName.
This approach also works very well when importing large numbers of MemberGroups:
foreach (MemberGroup memberGroup in memberGroupManager)
{
cachedMemberGroups.Add(memberGroup.DisplayName, memberGroup.Id);
}
NOTE: If you are wondering why not simply cache the entire UserProfile in the Dictionary (Dictionary<string, UserProfile>), the memory usage for this will be much higher, which will undo any gains by avoiding UserExists().
The UserProfileManager’s CreateUserProfile() method internally calls the UserExists method, and then calls an internal constructor on the UserProfile object to actually create the profile. By using reflection, you can call this internal constructor yourself and avoid UserExists():
// Get some reflected information about the UserProfile object for later use
ConstructorInfo ci = typeof(UserProfile).GetConstructor(
BindingFlags.NonPublic | BindingFlags.Instance,
null,
new Type[] { typeof(UserProfileManager), typeof(string), typeof(string) },
null);
Once you’ve got the reflected information, you can use the following code to create your UserProfile:
if (cachedProfiles.ContainsKey(accountName))
{
// Get existing profile...
}
else
{
// Create new profile
UserProfile newProfile = (UserProfile)ci.Invoke(new object[] { profileManager, accountName, displayName });
}
NOTE: I’ve tried creating a user profile in this manner that already existed to see what would happen. The existing profile was updated, and I did not get any duplicate records in the SharePoint db. It appears the SQL under the hood already takes care of avoiding duplicates. General cautions about reflection still apply here though (API may change, etc.).
Instead of pulling a huge recordset into a DataTable, DataSet, or into a collection of custom objects, try to process your records one at a time using a data reader if your data source permits. This will keep memory usage down, as the garbage collector will dispose frequently any variables you create within a while(reader.Read()) loop. A DataTable with 1 million records in it will take up tons of memory on top of the large memory consumption that OWSTIMER.exe does already.
using (OracleConnection conn = new OracleConnection(_connectionString))
{
using (OracleCommand cmd = new OracleCommand(_sqlGetAllUsers, conn))
{
conn.Open();
OracleDataReader rdr = cmd.ExecuteReader();
if(!rdr.HasRows())
{
return;
}
while (rdr.Read())
{
UserProfile profile = null;
string accountName = rdr["ACCOUNT_NAME"] as string;
string firstName = rdr["FIRST_NAME"] as string;
string lastName = rdr["LAST_NAME"] as string;
if (cachedProfiles.ContainsKey(accountName))
{
profile = profileManager.GetUserProfile(cachedProfiles[accountName]);
}
else
{
profile = (UserProfile)ci.Invoke(new object[] { profileManager, accountName, displayName });
cachedProfiles.Add(accountName, profile.ID);
}
if (!string.IsNullOrEmpty(firstName))
{
profile["FirstName"].Value = firstName;
}
if (!string.IsNullOrEmpty(lastName))
{
profile["LastName"].Value = lastName;
}
// ...
profile.Commit();
}
rdr.Close();
rdr.Dispose();
}
}
Of course other best practices also apply, such as:
Mar 04 2009
Recently I saw an access denied error when a user tried to edit a SharePoint Designer workflow task. The error did not occur if the user was a site collection administrator, but did occur even if they had full control to the site, list, and task list.
Running through the Request Access pages, it appeared that SharePoint did not think the user had rights to the task list, even though they did after thoroughly checking permissions.
I figured that it was having trouble with permissions to the ASPX pages from the Designer-generated tasks forms, so I solved it by doing the following:
It turns out that this “Workflows” node is actually a SharePoint folder object, and it turned out that this folder did not inherit permissions from the parent. The web page for managing permissions that came up showed me this, and enabled me to reinherit permissions and fix the problem.
I’m not sure why that folder didn’t inherit permissions though, I’d never seen that before.
Toth on Twitter