QueryOverride and its quirks - use with caution!

This post deals with the quirks involved with the QueryOverride property in a Content Query Editor WebPart. Suffice it to say when they say "override" they mean it. It overrides almost everything you set for a Content Query Editor Web Part in your UI and if you are unprepared this is a massive PITA to find and fix. I was, naturally, unprepared and had to find and fix it.

It started with a request to filter an existing UI so that news items lingering longer than a month would not be displayed. A bit of digging and I soon learned that the "Modify Shared Web Part" in the UI cannot calculate back from [Today], only a fixed date, so I would have to select Export on the menu and save to my own hard drive to edit the CAML in notepad. Naïve as I was, I thought it would be simple enough to replace the QueryOverride line with:

<property name="QueryOverride" type="string" ><![CDATA[<Where><And><Geq><FieldRef Name='Modified'/><Value Type="DateTime"><Today OffsetDays="-30"/></Value></Geq></Where>]]></property>

OK. Well I uploaded it to the site collection web part gallery, recreated my web part on the page to point to it, and that seemed to work. Last April's stuff was no longer appearing on the list. Except that one of my colleagues mentioned, "Hey, the order looks funny". I had a look and indeed she was right. Instead of descending order, it went from earliest first. So I went "Edit Page" and modified the web part to switch from Ascending to Descending, clicked "Apply" and hey presto…it switched right back to "Ascending" again! I tried this several times, with the same result. More googling and nail biting. Eventually I found that I needed to specify not only WHAT I wanted back, but what order I wanted it in. So my QueryOverride line became:

<property name="QueryOverride" type="string" ><![CDATA[<Where><Geq><FieldRef Name='Modified'/><Value Type="DateTime"><Today OffsetDays="-30"/></Value></Geq</Where><OrderBy><FieldRef Name="Created" Nullable="True" Type="DateTime" Ascending="FALSE"/></OrderBy>]]></property>

RIGHT. Upload, delete web part, recreate. So now we had the correct items appearing in the correct order. Happy Days. Except then the user said, "Hey, we have another problem".

The CEWP I was editing was pulling all items from all site collections on the portal that had a particular page content type. One library had items which were not to be included in the main list. It had a different page content type. Everyone knew about this other page content type and used it. The main news webpart had always ignored it because it was pointing to a different content type. Everything had been fine. Up till the time I'd put in my QueryOverride property, when it had all gone to dog doodoo. The items from the not-to-be-included library were very much included, and what was I going to do about it, eh?

I looked at the CAML of the two web parts, the good one and the bad one (i.e. mine)  - I copied them both into Excel and eyeballed them there, for God's sakes - and could not see what the problem was. Both web parts had their respective content types clearly specified in the Presentation section. There could be no ambiguity. So what was going on?

I finally discovered that QueryOverride doesn't care what you put in your content types. Once I put in the "Modified" date in the queryoverride, I was toast. It just hoovered up everything from everywhere REGARDLESS of what I specified for content types in the UI and what was written in the CAML. I realised I would have to tell the bloody thing for once and for all - don't use that &*(^ing content type. So now here is the final all-singing, all-dancing QueryOverride line I had to use. Oh and note the location of the <And> tags. I stuffed that up the first time and wondered why the logic wasn't working...

<property name="QueryOverride" type="string" ><![CDATA[<Where><And><Geq><FieldRef Name='Modified'/><Value Type="DateTime"><Today OffsetDays="-30"/></Value></Geq><Neq><FieldRef Name="ContentType"/><Value Type="Text">[My Little Content Type]</Value></Neq></And></Where><OrderBy><FieldRef Name="Created" Nullable="True" Type="DateTime" Ascending="FALSE"/></OrderBy>]]></property>

Please note that I have put the content type name in square brackets. I don't think my employer would care for me to put in the real content type name!

All right, good to go at last, surely! But no, not yet. The news items weren't displaying the way they were in the last web part, where we'd seen a nice little teaser under the headline. So I stuck the thing into Excel (again) to compare and amended the following CAML parameters:

<property name="ParameterBindings" type="string" null="true" />

<property name="CommonViewFields" type="string">PublishingPageContent</property>

<property name="NoDefaultStyle" type="string" null="true" />


Finally! It worked!

The All-Singing, All-Dancing Site Collection Recursion Code!

Need to update some setting in every site in your site collection? Hunting around the internet googling "Recursion", "Site Collection" and "SharePoint"? Here I am to your rescue. Create a console application, add in a ref to Sharepoint and replace the one line in Main() with your site collection. Stick your own code into the bit where it says "Put your code here", run, and Bob's your uncle, auntie and whatever sort of relative you're having yourself. Can be adapted for web part too if you want. The streamwriter dumps a load of updates into your chosen folder.

Why, you're welcome :)

//Code by Susan Lanigan, 25/10/2011

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.SharePoint;
using System.IO;



namespace RecursiveConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {

            //call once and let recursion take care of the rest
          //start from a site collection top level URL
            //NOTE - if you want to break completely on an error
            //stick a try-catch block up here and get the sub to throw;
            DoTheWork("http://myServer/mySiteColl", string.Empty);


        }

        static void DoTheWork(string siteUrl, string parentWebName, string eMail)
        {
            string errorArgs = string.Empty;

            try
            {

                using (SPSite site = new SPSite(siteUrl))
                {
                    using (SPWeb web = site.OpenWeb())
                    {
                        // another try-catch for the filestreamer
                        //write console output to file
                        FileStream fs = new FileStream("c:\\Susan\\LogForSite_" + parentWebName + "_" + web.Name + ".txt", FileMode.Create);
                        StreamWriter sw = new StreamWriter(fs);
                        TextWriter tmp = Console.Out;

                        try
                        {

                            Console.SetOut(sw);
                            Console.WriteLine("Writing update for site: " + web.Url);
                            //DO YOUR WORK HERE AND WRITE TO STREAMWRITER ETC.

                            //...

                        }
                        catch (Exception fileEx)
                        {
                            Console.WriteLine(fileEx.Message);
                            return;
                        }
                        finally
                        {
                            sw.Close();
                            fs.Close();
                        }

                        //reset console output
                        Console.SetOut(tmp);

                        //now for the subsites
                        SPWebCollection subSites = web.Webs;

                        foreach (SPWeb subSite in subSites)
                        {
                            //RECURSION HERE
                            DoTheWork(subSite.Url, web.Name, eMail);
                            subSite.Close();
                        }

                    }



                }

            }



            catch (Exception ex)
            {
                //custom error handling stuff
                Console.WriteLine(ex.Message);
                Console.ReadLine();
               
                return;
            }

        }
        }

}

Copying File from one Doc Library to Another - Problem Solved

I forgot to update about this. I solved that problem that had me removing several hairs at the follicle.

In order to copy the file from one place to the other, I had to create a using block for the source site and then the target site. Thus (note that this was in a separate class file from the event handler so I did not have access to the properties object and had to pass the IDs in as parameters):

public static void CopyFile (Guid listID, int itemID, string sourceUrl, string targetUrl, Guid targetListID)
{
     try
     {     
         using (SPSite sourcesite = new SPSite(sourceUrl))
         {
             using (SPWeb sourceWeb = sourceSite.OpenWeb())
             {
                  SPList list = web.Lists[listID];
                  SPListItem item = list.Items[itemID];
                  SPFile file = item.File;
                  //Here was where my error started
                  //I instantiated the file out of scope
                  byte[] binFile = file.OpenBinary();
                  
                  //OPEN TARGET SITE AND WEB HERE
                  using (SPSite targetSite = new SPSite(targetUrl))
                  {
                       using (SPWeb targetWeb = targetSite.OpenWeb())
                       {


                           //get the target list
                           //create SPFolder object
                           //this is like a SPList object
                           //but concerned only with the copying of files
                           folder.Files.Add(binFile);
                       }
                  }

     
             }
         }
     }
}


My error appeared to be where I declared the binary array. When I declared it within scope of the target site everything appeared to work perfectly, even though I was just streaming the file on the other site like before. I have no idea why a binary stream should lose scope, it's only an array, but I could not make it work for a non-admin user.


Please note that I haven't put in the target file copying code in detail as I am working from memory here - but it needs to be done using the SPFolder object. The SPFolder objects points to the document library just like the SPList object, but only deals with the files. Because the SPList object only handles stuff on lists. It doesn't do files. Jobsworth of an object!

Extending and Overriding my own Wits

Well I am making some progress on that non-event from hell detailed in my last post. First I commented out everything in my event handler and put in some innocuous line like:

properties.ListItem("Title") = "Changed by the event handler";
properties.ListItem.Update();


And that worked. So my fears that the event handling code was not invoked at all when logging in as a non-admin, nothing special user were unfounded.


So, I commented back in the error handling code. Nothing happened. OK, so the error handler doesn't work. At least that explains why I wasn't getting any feedback. Strange given that the test user had access to the folders but not crucial but will worry about it later.


So what did I do then? Moved the comment /* indicator down a few painstaking lines each time, compiled, GACed, IISResetted and ran. Used the Title field as my error handler! Bit by bit by bit. The code had no problem calling the web service or using the impersonator token. What it did not want to do was copy the binarily streamed file (is that valid syntax? Probably not, but if the method of streaming a file is binary, then surely it is binarily streamed, no? Anyway.) It streamed it fine, even modified as Mr High-Up Impersonated User just as I asked it to with the SPUserToken. But it won't copy it.


I suspect I will have to put a Special Snowflake sodding privilege block around that operation all by itself or else it will sulk forever. So, will I succeed. Tune in...to be continued in our next. If you haven't died of boredom first...

At My Wit's End

I cannot get an event handler to fire at all for a non-admin user.

I have error tracking enabled  and on. It writes to a file. I have enabled permissions on the file folder. I have instructed it to throw an error and write back. When I log in as myself, it does all these things. I am a member of the owner group of the site.

The event handler copies the document library entry, including the document, to another document library on a different site, in a different site collection. It does various other things as well, but that's later on. When I log in as myself, all this works fine. When I log in as the test user, nothing happens. When I instrument the code, nothing happens. I have tried elevated privilege blocks in the code, paying attention to the declaration of the site instances within the block as required, but given that the event handling mechanism steadfastly refuses to even CALL the code for that user, I'm inclined to believe I could put "rhubarb rhubarb rhubarb" in there and it wouldn't make any difference. There should be no reason why I cannot manipulate the properties object.

I made the test user King of Everything. Full control on the source document library, full control on the site - I even made the blasted thing a site collection administrator! (this on the test machine, of course) Test user had more privileges than I did. Plus I removed my login from the site collection admin list.

Didn't make a lick of difference. And then when an actual user tried to log in and nothing happened, I knew this was going to be trouble. I have to confess to being utterly stumped as to why it doesn't call the code AT ALL. I would understand if it gave an error due to privilege - I even managed to get past one of those in one of the other issues I had with the EnsureUser method - but it's not even bothering to invoke the component. Works fine when I log in as myself, or log in as super user.

Any ideas? Losing the will to live over here! :)

Susan

RunWithElevatedPrivileges - another encounter and something you should know

I have been previously complaining about SPSecurity.RunWithElevatedPrivileges as being a load of tosh. I might have been a tad harsh as I employed it just yesterday and got it working - almost.

I needed to do two things - copy over a file and update fields. The file copy bit worked well - I had an impersonation block for that - but the bit where a non-admin user updates the fields was, alas, a bit of a dud. Until I put the field update section into a RunWithElevatedPrivilege delegate block and all looked fine.

But in addition to the native data types I was using, I also had four user fields I was obtaining from a web service off somewhere else. In order to update these, I had to get the username and call the EnsureUser method on my elevated privileges SPWeb object. This was still failing, even within the privileged user wrapper. It either said I couldn't find the user or that I wasn't allowed to get it. This even though I had, as Google's wisdom advised, wrapped the EnsureUser method call with AllowUnsafeUpdates = true on either side.

Then I commented out the web.AllowUnsafeUpdates = true line and its corresponding AllowUnsafeUpdates=false on the other side. Ran it again. Worked perfectly. I had been trying this for days!

So, you heard it here first - when using the delegate, try not setting the AllowUnsafeUpdates property. And definitely don't set it on the Site object or the whole thing will break completely. SharePoint seems perfectly capable of deciding for itself, in this case, whether the update is safe or not.

Programmatically Copying a Recurring Event From One Calendar to Another

I have been spending the last while trying to figure out a problem I had with an event handler I had configured for a calendar. The idea is that if you add an event to a calendar where the event handler is switched on, it bubbles up to a parent calendar which contains all the events. There can be many sub-calendars, only one main one.

So, I created my event handler just as I have described in previous entries. Create an event on the child, bubbles up to the parent. Create a common key between the two. Add an ItemUpdated event handler so then if the child event is updated the parent is updated too. Same with ItemDeleted. Everything hunky dory -

OK, wait, back up the truck. What about recurring events? How do they work?

Leaving them out was not an option. The department had quite a few weekly meeting events. So I hit Mr Google and got a handy link from MSDN here which suggested the following code.

SPListItem recEvent = listItems.Add();

string recData = "<recurrence><rule>" + 
    "<firstDayOfWeek>su</firstDayOfWeek>" +
    "<repeat><daily dayFrequency='1' /></repeat>" +
    "<repeatInstances>5</repeatInstances></rule></recurrence>";

recEvent["Title"] = evtTitle;
recEvent["RecurrenceData"] = recData;
recEvent["EventType"] = 1;
recEvent["EventDate"] = new DateTime(2011,8,15,8,0,0);
recEvent["EndDate"] = new DateTime(2011,9,25,9,0,0);
recEvent["UID"] = System.Guid.NewGuid();
recEvent["TimeZone"] = 13;
recEvent["Recurrence"] = -1;

recEvent.Update();

Now obviously I need to change the line where they build the RecurrenceData property because this will depend on whatever that is for the original object. So the line to use will be

item["RecurrenceData"] = properties.ListItem["RecurrenceData"];

and for the same reason I don't need to code the UID. So I save it and all is well. Then I a dd the recurring event. Let's say I set the start time at 1pm, the end time at 2pm, set it to occur weekly on a Tuesday for two weeks, and save it. Then I go to my parent calendar. In spite of the MSDN advice, still no sign of the recurring event being copied over.

So, I turn on the error handler and get back this error: Value does not fall within accepted range. This error means that the field RecurrenceData does not exist on the list. But how can it not exist? The Recurrence field exists for every event, surely?

Ah - but they are only visible on the form if you click the little Recurring Event checkbox! So if they are not visible on the form, some little quirk means that they are not visible in the code. This is probably because one recurring event contains many child events. So after much searching around I found this link which explained that I had to set the ShowInEdit property of the field concerned to true:


item.Fields["Recurrence"].ShowInEditForm = true;

The only thing that did not work for me in that link was the use of the GetInternalFieldName method. I replaced that with the GetField method, used those lines of code before updating each recurrence data field - you need to do it for all of them and don't forget to call this.DisableEventFiring() after each update! - and then built it once more. Hey presto, it created an event - of sorts.

The problem I have now is that the event that gets created on the parent calendar is a long, unspecified blob that covers all the days of the recurrence time. In order to have it work properly, I had to go into the event from the UI and click "Edit Item". The recurrence info is all there, present and correct. Click Ok, click past the warning message, and it's fine. All I need to do is mimic that update in code. All ideas and suggestions welcome!

ETA. See the following solution kindly provided by Noelle Marchbanks

Add to your copy code:
recEvent["EventType"] = properties.ListItem["EventType"];
recEvent["UID"] = System.Guid.NewGuid();

Even though you are copying an event, these fields are necessary to make it realize that a Recurrence is appropriate. It is IMPERATIVE that you set the EventType to the same as the copied list item, and not hard-code it as 1 as it will break non-recurring events and create orphaned exceptions to the rule.

If, for some reason, you do get orphaned recurring event children (e.g. ID = 1.1.x instead of ID = 1), open up powershell, get the item as an object, set its EventType to 1, then delete it from the list.

Hope this helps.