Skip to content
/

When writing .NET code to access a SQL database we often rely on the Entity Framework (EF). The EF makes it very easy to retrieve data from the database by generating a SQL Query for us. But we shouldn't trust it blindly as the EF can also generate a bad query. It will return the […]

When writing .NET code to access a SQL database we often rely on the Entity Framework (EF). The EF makes it very easy to retrieve data from the database by generating a SQL Query for us. But we shouldn't trust it blindly as the EF can also generate a bad query. It will return the correct data yes, but at what performance cost?

I have a table with lots of data, and to keep the table as small as possible on disk and in memory I want to optimize the columns. This can be done by using varchar instead of nvarchar, date instead of datetime and also using tinyint instead of int. Of course you only can do this if your data allows this.

The problem

Changing a column in my table from int to tinyint gave me a storage and memory space win, but it also gave me a performance loss!

In my case the SQL column "HasSong" is a tinyint. EF translates this to a Byte Structure in the .NET model.

The following code:

var query = from d in dataEntities.Donations
            where d.HasSong == 1
            select d.DonationId;

produces this SQL Query:

SELECT
    [Extent1].[DonationId] AS [DonationId]
FROM
    [dbo].[Donations] AS [Extent1]
WHERE
    1 = CAST([Extent1].[HasSong] AS int)

When SQL encounters this CAST it will skip all Indexes that are on the HasSong column resulting in the query engine using non-optimized indexes or even worse: full table scans.

So this explains my performance loss, but how do we convince EF not to cast my byte to an int?

The familiar "Contains"

Browsing the internet gave me my first idea: using the familiar contains method I encountered on an earlier post about the EF: Joining an IQueryable with an IEnumerable.
I just have to add an Array with a single value.

So trying this:

var tinyintComparison = new byte[] { 1 };

var query = from d in dataEntities.Donations
            where tinyintComparison.Contains(d.HasSong)
            selectd.DonationId;

was not such a good idea as it throws an ArgumentException:
Screenshot showing Visual Studio debugger with the message "ArgumentException: DbExpressionBinding requires an input expression with a collection ResultType"
It did work for other people using a SMALLINT, so I guess an array of bytes is a special case reserved for working with binary data in your database.

The solution

Lucky enough we are not far from a solution, changing the array to a List<T> Class:

var tinyintComparison = new List<byte> { 1 };

var query = from d in dataEntities.Donations
            where tinyintComparison.Contains(d.HasSong)
            selectd.DonationId;

results in the following query:

SELECT
    [Extent1].[DonationId] AS [DonationId]
FROM
    [dbo].[Donations] AS [Extent1]
WHERE
    1 = [Extent1].[HasSong]

There we are! No more casting. The SQL query engine can use the correct indexes again and we still won disk and memory space.

Conclusion

If you don't want a CAST in your SQL query when using a tinyint use the IList<T>.Contains Method.

Filed under C#, SQL
Last update:
/

Recently I was challenged with the task to set the layout and content of a wiki page when a new page is added to a team site. As I'm used to work with SharePoint publishing the task sounded easy, but I was wrong.

Recently I was challenged with the task to set the layout and content of a wiki page when a new page is added to a team site. As I’m used to work with SharePoint publishing the task sounded easy, but I was wrong.

Text Layout

Image showing the Text Layout option in the SharePoint ribbon

My first path was to figure out where SharePoint puts the wiki “text layouts”. I discovered this isn’t how it works. The layouts available for wiki’s are not configurable anywhere.

But using some PowerShell it was easy to get and set the layout as it’s the HTML content of the “Wiki Field” column in the list.

$web = Get-SPWeb http://server/teamsite
$list = $web.Lists["Site Pages"]
$listItem = $list.Items[2] # My test page
$listItem["Wiki Content"] # Returns HTML

The HTML content consists of 2 parts. The layout table and the layout data.

<table id="layoutsTable" style="width: 100%">
    <tbody>
        <tr style="vertical-align: top">
            <td style="width: 100%">
                <div class="ms-rte-layoutszone-outer" style="width: 100%">
                    <div class="ms-rte-layoutszone-inner">
                    </div>
                </div>
            </td>
        </tr>
    </tbody>
</table>
<span id="layoutsData" style="display: none">false,false,1</span>

The layout data describes visibility of the header and footer and the number of columns.

Event receiver

To set the content the first thing in my mind was to add an , associated with ListTemplateId 119 (WebPageLibrary).

I deployed the solution and added a page and tadah: no content!

Using the debugger to verify my event receiver was triggered, I went to the next option: adding an . This time I got an exception the page was already modified by another user. Refreshing the page gave me the default content. So this told me 2 things:

  1. It’s possible to set default content
  2. I forgot to set the Synchronize property

So fixing the second thing I deployed once again and got: no content!

As I used in my receiver I got a version history where it showed the content was set, but the final version still ended up empty.

When faced with utter desperation, working with SharePoint has taught me you always have an escape: launch Reflector.

There I found this gem of code in the SubmitBtn_Click method of the CreateWebPage Class:

SPFile file = SPUtility.CreateNewWikiPage(wikiList, serverRelativeUrl);
SPListItem item = file.Item;
item["WikiField"] = "";
item.UpdateOverwriteVersion();

So no matter what I do in ItemAdding or ItemAdded, the content always ends up empty!

After this discovery, the fix was removing the code from the ItemAdding and ItemAdded events and moving it to the event (synchronious) and added a check if the WikiField content is an empty string.

public override void ItemUpdated(SPItemEventProperties properties)
{
    base.ItemUpdated(properties);

    var listItem = properties.ListItem;

    if (!string.IsNullOrEmpty(listItem["WikiField"] as string))
    {
        return;
    }

    this.EventFiringEnabled = false;

    listItem["WikiField"] = html;
    listItem.UpdateOverwriteVersion();

    this.EventFiringEnabled = true;
}

Now every wiki page I add to the team site contains the correct text layout and contains the default HTML.

/

I’ve been running my own mail server at home for years. But it requires a reliable connection and some maintenance once in a while. And of course it always breaks when I‘m on the other side of the world.
To free myself of that burden I decided to make the move to Office 365. However I discovered there is no way to set my account as a catch-all account. This is not possible at all!

So I made my own scripts to add all email addresses I used in the past as an alias on my mailbox.

I’ve been running my own mail server at home for years using Postfix, dovecot, amavisd-new, ClamAV and SpamAssassin. But it requires a reliable connection and some maintenance once in a while. And of course it always breaks when I’m on the other side of the world.

To free myself of that burden I decided to make the move to Office 365. I got myself a P1 subscription and started to browse through the configuration screens. The migration of an account from IMAP to Exchange Online was very fast and easy.

Happy with how everything looked, felt and connected, I was ready to make the switch.

Just before I wanted to change the MX record to point to Office 365 I double checked the configuration of my account. I discovered I couldn’t find a way to set my account as a catch-all account. After some research I found out this is not possible at all!

Catch-all Mailbox

A catch-all mailbox receives messages sent to email addresses in a domain that do not exist. Exchange Online anti-spam filters use recipient filtering to reject messages sent to mailboxes that don’t exist, so catch-all mailboxes are not supported.

That left me 2 options:

  1. Stop the migration to Office 365, and leave things as they were.
  2. Make every email address I used in the past an alias.

I started searching if anyone has done this before. It looks like this is not the case, so seeing this as a challenge, I started working on my own solution.

Extracting all used addresses

First you need to get every email address ever used by others as a recipient.

As my bash scripting is a bit rusty, I found this by Joerg Reinhardt which I used as base for my own script: getting email recipient addresses from maildir.

The script needs 2 parameters, the maildir directory and the email domain you want to get the aliases for.

$ ./maildir-dump.sh <Maildir directory> <Email domain>

So for me:

$ ./maildir-dump.sh Maildir hompus.nl

This results in a long file with a lot of email addresses. To aggregate this list into a CSV file you can use the following command:

sort Maildir.dump | uniq -c | sort -k1nr | awk 'BEGIN {OFS = ";";} {print $1,$2}'
                                                                          > Maildir.csv

This allows you to open the file using Excel and remove all entries you don’t want to be an alias.
I counted 385 unique email addresses, too many to add manually.

Adding the aliases to a mailbox

First I configured and connected to Exchange Online using the article.

First we will read the CSV file we have generated and authored.

$csv = Import-Csv D:\Maildir.csv -Delimiter ';' -Header Count, Email

Then you need to get a reference to the mailbox you want to add the aliases to.

$temp = Get-Mailbox -Identity michael

You need to add all email addresses to the EmailAddress property, but I discovered the last email address to be added will become the default address. So make sure the primary SMTP address is added last.

$temp.EmailAddresses = $temp.EmailAddresses | ? { $_ -ne "SMTP:" + $temp.PrimarySmtpAddress }
$csv | % { $temp.EmailAddresses += ("SMTP:" + $_.Email) }
$temp.EmailAddresses += "SMTP:" + $temp.PrimarySmtpAddress

Now you only need to set the EmailAddress property on the actual mailbox

Set-Mailbox -Identity michael -EmailAddresses $temp.EmailAddresses

And done!