Web Request Always Redirects, While Web Browser Does Not

Today I had a breakthrough on an issue that has stewing for many months.  Every once in a while, I would pick the project back up again and see if I could approach it differently and maybe come up with a solution.  Today is the day.

The problem was in one of my homebrew personal utility programs, used to download images from the Internet.  One day, when I went to run the utility, it suddenly started failing and I could not get it to resume.  When I would request the same URL in a web browser, the download worked without any problem, but trying to download using code was a no-go.

Clearly, there is a difference between handmade code and a web browser, the task was to determine what the difference was in order to make the code better emulate the browser.  I added many http headers, trying to find out what was the missing piece.  I tried capturing cookies, but that had no effect either.  I even used a network monitor to capture the network traffic and couldn’t see any significant difference between my code and a browser.

Today, starting from scratch in a new project, I tried again.  To get the basic browser header that I needed, I visited a web site, like I had done in previous attempts.  This particular web site returned a number of headers I had never seen before, which I found very curious.  When I duplicated all those headers in my code, my requests worked.  Eliminating the headers one by one identified the one that I needed: X-Forwarded-Proto.

This header has been around for nearly a decade, although I’ve never heard of it, nor seen it implemented anywhere.  It seems to be specific to load balancers, which is actually relevant since the image host I am calling runs through a load balancer.  I was able to add the header to a standard WebClient and didn’t actually need to go to the HttpWebRequest level of code and my requests started working again.

I am not sure if the obscurity of this header was a browser-sniffing check by the load balancer to weed out bots and automated download tools (of which my utility would be classified), or whether it was just part of a new, stricter set of w3c standards that got included in a software update.  Realistically, it didn’t break anything for their users since their primary traffic is web browsers.

Utility: HardLinker

Many years ago, I used to use ACDSee for image viewing and have since switched to Faststone Image Viewer.  Not all that long ago, I had sent an email to their “comment box” with praise and a suggestion (the praise was to hopefully get my suggestion implemented).  Despite not hearing back from them, it didn’t stop me from brainstorming solutions to my problem.  I have found a way to address my needs, by writing a utility.

To summarize my request, I was looking for some sort of photo management like “albums” or “collections”.  For a pure image viewing application like Faststone, I can understand the trepidation the developers would have in implementing that feature.  It would be sure to divide their userbase as to whether it was done properly and whether it should have been done at all.

You can organize photos into folders on your hard drive, sure, but what if you want a photo in two or more albums?  You have duplicate files wasting space and possibly getting out of sync.  So my suggestion was that the file browser in FastStone have the capability to parse windows shortcuts, which are just LNK text files with pointers to the original file.  That way, you could put shortcuts into album folders and keep the originals where they are.

Recently, I stumbled on the concept of symbolic links, which are like shortcuts, but operate at a lower level.  A symbolic link looks just like a file to an application.  It doesn’t know any different.  By using symbolic links, I could get the functionality I needed with no application changes to Faststone.  I tested this concept and it was completely successful.  So I set about trying to figure out how to build something that I could use with Faststone.

Faststone has a way to call external applications for editing photos.  I could use this same feature to call a utility to generate symbolic links in a selected folder.  My first version of the utility was very simple.  It simply opened a FolderBrowserDialog and wrote links for the files specified on the command line.  However, FastStone had a limitation.  It would not send multiple filenames on a command line.  Instead it would call the external application over and over, once per selected file.

The second version of the utility would capture the multiple calls and write the links in one pass, as long as you left the dialog open until Faststone was done calling the external application.  This was ok, but I couldn’t get any visual feedback on how many files were queued up while the dialog was open.  So I had to create my own folder browser and use my own form which would not be open modally.

As with any attempt to recreate a standard windows control. the results are less than perfect, although serviceable.  The result is a utility called HardLinker.  HardLinker will remember the last folder you used as well as the size of the window.  The files queued will climb as the utility is called multiple times.

image

Under normal usage, you need to be an administrator to create a symbolic link, so HardLinker needs to be run as an administrator.  However, in Windows 10, now you can enable developer mode and create links without being in admin mode.  See: https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/ for more information.

So, in addition to calling HardLinker from Faststone, I also added a shortcut to the utility in my SendTo folder, so I can use it with Windows Explorer.  Calling it from Explorer is very fast since all the files are on the command line and you don’t have to wait for the multiple instances to keep spinning up.

If you are interested in this utility, you can download it from here.

CUERipper CUE Only Modification

In another hobby of mine, CD collecting, I utilize a few different tools in my collecting and cataloging process.  One of which is Exact Audio Copy (EAC), which I use to rip the CD audio to my computer.  When you collect early-edition CDs as I do, you might run into some CDs that had a special equalization called pre-emphasis, which requires you to apply an inverse equalization to the files after ripping.  Additionally, some early CDs had track subindexes, where you could jump to defined sections within a long track.

EAC, while being a good ripping program, is not able to identify either of these special features, which is why I also use CUERipper.  CUERipper is a part of the CUETools suite, which is an open-source program for ripping, authoring, and modifying CUE files.  CUE files are the true index of a CD’s tracks.  They include information such as indexes, track gaps, and pre-emphasis flags.

However, one of the drawbacks to CUERipper is that you have to rip the entire CD to get the CUE file.  Well, you can cancel the rip early, or you can specify an invalid compressor causing it to error out after the CUE generation.  Neither is an efficient workflow.  So, I downloaded the source code and made some changes specific to what I wanted.

Since I was going to run through a large number of CDs to get the pre-emphasis and index info, I wanted a way to only generate a CUE file.  To make the process even quicker, I wanted CUERipper to start the rip as soon as it had the album metadata to use, and I wanted it to eject the CD as soon as it was done with the CUE file.

image

I added three new options to CUERipper, with obvious meanings:

  • Eject after extraction
  • Start immediately
  • Create CUE file only

The options are useful even if you are doing a full CD rip, but for my needs, the CUE file generation was reduced to inserting a CD, waiting about 30 seconds while the disc indexes are read, then switching CDs after the tray ejects.

If you are interested in using this modified version of CUERipper, you can download it from here.

Youtube-dl Bookmarklet

One of my favorite things is custom protocol handlers.  The idea you can launch an application with a URL is awesome to me.  I implemented it at my current company for their internal application and it is far and away the most progressive feature the business users love.

One of my side hobbies is digital archiving, which is simply downloading stuff and saving it.  Aside from the archivist mentality of it, I do like to have local copies of items just because you can’t trust the Internet for anything.  Content comes and goes at will, and even with that, connectivity comes and goes, and one thing that keeps coming and never goes is advertising.  So, local copies win out every time in every case.

I’ve written utilities to assist in my image archiving, but when it comes to video archiving, I’ve usually used FreeDownloadManager. but sometimes it doesn’t do the job.  I had recently learned of an open-source tool called YouTube-dl, which is a command-line tool to download videos from a wide variety of websites.

I downloaded it and tried it out on a few concert videos and the utility worked quite well, so it’s been added into my toolkit.  But, being a command-line tool, it’s a little cumbersome.  What I wanted to do was launch the downloader passing the URL of the page I was currently viewing, which was hosting the video I wanted to download.  This is something I do with my image downloading utilities, too, so it’s a concept not too foreign to me.  It involves custom protocol handlers and bookmarklets.

First thing we need to do is create a custom protocol handler.  For this, ytdl:// is the protocol I chose to use.  I created a registry file (protocol.reg) with the following contents.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\ytdl]
"URL Protocol"=""
@="URL:Youtube-DL Application"

[HKEY_CLASSES_ROOT\ytdl\shell]
[HKEY_CLASSES_ROOT\ytdl\shell\open]
[HKEY_CLASSES_ROOT\ytdl\shell\open\command]

@="\"wscript \"y:\\videos\\youtube-dlWrapper.vbs\" \"%1"\""

Notice the execution path command.  This will obviously be different paths for different people, but there is a VBScript file that needs to be created at that location to execute.  The contents of that script are as follows.

dim shell
set shell=createobject(“wscript.shell”)

cmdline=”””y:\videos\youtube-dl.exe”” “”” & replace(wscript.arguments(0),”ytdl://”,””) & “”””
ret=shell.run(cmdline,8,true)

set shell=nothing

The cmdline is generated to execute the actual youtube-dl utility, passing the URL of the video.  However, the URL in arguments(0) has the custom protocol handler ytdl:// prefixing it, so that is stripped out.

The last part is the bookmarklet itself.  In your web browser, create a new bookmark for whatever page you’re on, then edit the bookmark to give it a better title and to have the following URL:

javascript:location.href='ytdl://'+location.href;

This is how the VBScript ends up with ytdl:// in the address.  To summarize the entire process:

  1. You click the bookmarklet on a page with a video
  2. The URL of the page is prepended with ytdl:// and is navigated to
  3. Your browser recognizes it as a custom protocol and executes the URL on the local system (after appropriate security warnings)
  4. Windows looks up the protocol in the registry and determines it is to launch youtube-dlWrapper.vbs passing the URL as an argument.  The URL still has ytdl:// in front of it at this time.
  5. The wrapper VBScript strips out the leading protocol and passes the remaining URL to the youtube-dl.exe for download.
  6. A command window opens with youtube-dl running and closes when the download is complete.

Batch Applying MetaFLAC ReplayGain Tags To Recursive Album Folders

I’ve been ignoring PowerShell for a very long time, even though I know it is the future of scripting.  The structure of the language is very foreign, with all of the pipe options.  But I muddled through the examples I found online and came up with my first usable PowerShell script.

The purpose of the script is to iterate recursively through a folder structure and generate a command statement using all the files in each folder.  In this case, MetaFLAC is being called with arguments to apply ReplayGain tags to all the files in a folder.  To do this effectively, you have to pass all the file names in the folder on one command line.  This is so the overall album sound level can be calculated.

Without further introduction, here is the script:

<# 
     This script applies album-level and file-level 
     ReplayGain tags to FLAC album folders recursively

    Example usage from PS command line: 
     .\ReplayGain.ps1 -root:"c:\music\FLAC"

    MetaFLAC.exe must be in same folder or in environment PATH 
#>

Param( 
     #Fully-qualified path to top-level folder. 
     [string]$root 
     )

Function CallMetaFLAC($d){ 
     Write-Host "Processing" ($d | Get-ChildItem -Filter "*.flac").length "files in" $d 
     if (($d | Get-ChildItem -Filter "*.flac").length -gt 0){ 
         $args="--add-replay-gain "

        foreach ($f in ($d | Get-ChildItem -Filter "*.flac" | % {$_.FullName}) ){ 
             $args=$args + """$f"" " 
             }

        Start-Process "metaflac.exe" $args -Wait -NoNewWindow 
     }

    # Process Subfolders 
     foreach($dd in $d | Get-ChildItem -Directory){ 
         CallMetaFLAC $dd 
         } 
}

Write-Host "Starting in $root" 
Write-Host

CallMetaFLAC $root

Write-Host 
Write-Host "Ending"

Arrangeable FlowLayout Panel

I found this piece of code and when I ran it, I was surprised how usable it actually was.

At the time, I needed something that would allow me to sort an array of things by dragging and dropping.  The things I was working with were photos, but this class handles pretty much everything.

You can grab an item and drag it into a new position.  A caret shows you where the new position will be.  Even better, you can multi-select items and drag them to a new position.  When you need the new order, just iterate through the child controls.

I don’t actually have a use for it right now, but I need to save this so I have it for the future.

Public Class ArrangeableFlowLayoutPanel
    Inherits FlowLayoutPanel

    Protected mouseDownPoint As Point
    Protected insertCaret As PictureBox
    Protected isMultiSelectOn As Boolean
    Protected isRangeSelectOn As Boolean

    Public Property AllowReordering As Boolean = True
    Public Property CaretColor As Color = Color.Green
    Public Property CaretWidth As Integer = 3
    Public Property CaretPadding As Padding = New Padding(2, 0, 2, 0)
    Public Property SelectionColor As Brush = Brushes.Black
    Public Property SelectionWidth As Integer = 1
    Friend Property SelectedControls As New Generic.List(Of Control)
    Public Property DragTolerance As Integer = 40

    Public Event ItemOrderChanged(sender As Object, e As EventArgs)

    Public Sub New()
        Me.AllowDrop = True
        Me.AutoScroll = True

        CreateCaret()

    End Sub

    Private Sub Form_Key(sender As Object, e As KeyEventArgs)
        isMultiSelectOn = e.Control
        isRangeSelectOn = e.Shift
    End Sub

    Private Sub ArrangeableFlowLayoutPanel_ControlAdded(sender As Object, e As ControlEventArgs) Handles Me.ControlAdded
        AddHandler e.Control.MouseDown, AddressOf Item_MouseDown
        AddHandler e.Control.MouseUp, AddressOf Item_MouseUp
        AddHandler e.Control.MouseMove, AddressOf Item_MouseMove
        AddHandler e.Control.Paint, AddressOf Item_Paint

    End Sub

    Private Sub ArrangeableFlowLayoutPanel_ControlRemoved(sender As Object, e As ControlEventArgs) Handles Me.ControlRemoved
        RemoveHandler e.Control.MouseDown, AddressOf Item_MouseDown
        RemoveHandler e.Control.MouseUp, AddressOf Item_MouseUp
        RemoveHandler e.Control.MouseMove, AddressOf Item_MouseMove
        RemoveHandler e.Control.Paint, AddressOf Item_Paint

        If SelectedControls.Contains(e.Control) Then SelectedControls.Remove(e.Control)

    End Sub

    Private Sub ArrangeableFlowLayoutPanel_ParentChanged(sender As Object, e As EventArgs) Handles Me.ParentChanged
        Dim f As Form

        f = Me.FindForm
        If f IsNot Nothing AndAlso Not f.KeyPreview Then f.KeyPreview = True

        RemoveHandler Me.FindForm.KeyDown, AddressOf Form_Key
        RemoveHandler Me.FindForm.KeyUp, AddressOf Form_Key

        AddHandler Me.FindForm.KeyDown, AddressOf Form_Key
        AddHandler Me.FindForm.KeyUp, AddressOf Form_Key
    End Sub

    Private Sub ArrangeableFlowLayoutPanel_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed
        insertCaret.Dispose()

        RemoveHandler Me.FindForm.KeyDown, AddressOf Form_Key
        RemoveHandler Me.FindForm.KeyUp, AddressOf Form_Key

    End Sub

    Private Sub ArrangeableFlowLayoutPanel_DragDrop(sender As Object, e As DragEventArgs) Handles Me.DragDrop
        Dim dropIndex As Integer

        For i As Integer = 0 To SelectedControls.Count - 1
            dropIndex = Me.Controls.GetChildIndex(insertCaret)
            Me.Controls.SetChildIndex(SelectedControls(i), dropIndex + 1)
        Next

        Me.Controls.Remove(insertCaret)
        RaiseEvent ItemOrderChanged(Me, New EventArgs)

    End Sub

    Private Sub ArrangeableFlowLayoutPanel_DragLeave(sender As Object, e As EventArgs) Handles Me.DragLeave
        Dim topBorderY As Integer
        Dim bottomBorderY As Integer
        Dim mousePositionY As Integer
        Dim hostForm As Form

        Me.Controls.Remove(insertCaret)

        hostForm = Me.FindForm
        topBorderY = hostForm.PointToClient(Me.Parent.PointToScreen(Me.Location)).Y
        bottomBorderY = Me.Height + topBorderY
        mousePositionY = hostForm.PointToClient(MousePosition).Y

        Do While mousePositionY >= bottomBorderY ' Below bottom of control
            If Me.VerticalScroll.Value <= Me.VerticalScroll.SmallChange + Me.VerticalScroll.Maximum Then
                Me.VerticalScroll.Value += Me.VerticalScroll.SmallChange
            Else
                Me.VerticalScroll.Value = Me.VerticalScroll.Maximum
            End If

            mousePositionY = hostForm.PointToClient(MousePosition).Y
            Me.Refresh()

        Loop

        Do While mousePositionY <= topBorderY ' Above top of control
            If Me.VerticalScroll.Value >= Me.VerticalScroll.SmallChange - Me.VerticalScroll.Minimum Then
                Me.VerticalScroll.Value -= Me.VerticalScroll.SmallChange
            Else
                Me.VerticalScroll.Value = Me.VerticalScroll.Minimum
            End If

            mousePositionY = hostForm.PointToClient(MousePosition).Y
            Me.Refresh()

        Loop

    End Sub

    Private Sub ArrangeableFlowLayoutPanel_DragOver(sender As Object, e As DragEventArgs) Handles Me.DragOver
        Dim ctl As Control
        Dim dropControlPosition As Point
        Dim dropIndex As Integer

        If e.Data IsNot Nothing Then
            e.Effect = DragDropEffects.Move
            ctl = Me.GetChildAtPoint(Me.PointToClient(New Point(e.X, e.Y)))

            If ctl IsNot Nothing AndAlso ctl IsNot insertCaret Then
                dropControlPosition = ctl.PointToClient(New Point(e.X, e.Y))

                If dropControlPosition.X <= ctl.Width \ 2 Then
                    dropIndex = Me.Controls.GetChildIndex(ctl) - 1
                Else
                    dropIndex = Me.Controls.GetChildIndex(ctl) + 1
                End If

                If dropIndex < 0 Then dropIndex = 0

                If Not Me.Controls.Contains(insertCaret) Then
                    insertCaret.Height = ctl.Height
                    Me.Controls.Add(insertCaret)
                End If

                Me.Controls.SetChildIndex(insertCaret, dropIndex)

            End If

        End If

    End Sub

    Private Sub Item_MouseDown(sender As Object, e As MouseEventArgs)
        If e.Button = System.Windows.Forms.MouseButtons.Left Then
            mouseDownPoint = e.Location
        End If

    End Sub

    Private Sub Item_MouseUp(sender As Object, e As MouseEventArgs)
        Dim ctl As Control
        Dim startIndex As Integer
        Dim endIndex As Integer
        Dim newCtl As Control

        If e.Button = System.Windows.Forms.MouseButtons.Left Then
            ctl = DirectCast(sender, Control)

            ' Choosing individual items or the first of a range
            If isMultiSelectOn OrElse (isRangeSelectOn And SelectedControls.Count = 0) Then
                If SelectedControls.Contains(ctl) Then
                    SelectedControls.Remove(ctl)
                Else
                    SelectedControls.Add(ctl)
                End If

                ctl.Invalidate()

                ' Choosing the end of a range
            ElseIf isRangeSelectOn Then
                startIndex = Me.Controls.GetChildIndex(SelectedControls(SelectedControls.Count - 1))
                endIndex = Me.Controls.GetChildIndex(ctl)

                For i As Integer = startIndex To endIndex Step CInt(IIf(startIndex < endIndex, 1, -1))
                    newCtl = DirectCast(Me.Controls(i), Control)

                    If Not SelectedControls.Contains(newCtl) Then
                        SelectedControls.Add(newCtl)
                        newCtl.Invalidate()
                    End If

                Next

            Else
                SelectedControls.Clear()
                SelectedControls.Add(ctl)
                For Each c As Control In Me.Controls
                    c.Invalidate()
                Next

            End If ' single or multi-select

        End If ' Left button only

    End Sub

    Private Sub Item_MouseMove(sender As Object, e As MouseEventArgs)
        Dim ctl As Control
        Dim rect As Rectangle
        Dim rectPoint As Point

        If AllowReordering AndAlso e.Button = System.Windows.Forms.MouseButtons.Left Then
            ctl = DirectCast(sender, Control)

            ' create a range before dragging activates
            rectPoint = New Point(mouseDownPoint.X, mouseDownPoint.Y)
            rectPoint.Offset(0 - (Me.DragTolerance \ 2), 0 - (Me.DragTolerance \ 2))

            rect = New Rectangle(rectPoint, New Size(Me.DragTolerance, Me.DragTolerance))

            ' See if we've dragged outside the tolerance area
            If Not rect.Contains(e.Location) Then

                ' dragged item is not in selection, include it if ctrl is held
                ' otherwise, clear the selection and only use the dragged item
                If Not SelectedControls.Contains(ctl) Then
                    If isMultiSelectOn Then
                        SelectedControls.Add(ctl)
                        ctl.Invalidate()

                    Else
                        SelectedControls.Clear()
                        SelectedControls.Add(ctl)
                        For Each c As Control In Me.Controls
                            c.Invalidate()
                        Next

                    End If ' Ctrl held down

                End If ' Not in current selection

                Me.DoDragDrop(SelectedControls, DragDropEffects.Move)

            End If ' Outside drag buffer area

        End If ' mouse button down

    End Sub

    Private Sub Item_Paint(sender As Object, e As PaintEventArgs)
        Dim ctl As Control

        ctl = DirectCast(sender, Control)

        If SelectedControls.Contains(ctl) Then
            ' Draw outline
            e.Graphics.DrawRectangle(New Pen(Me.SelectionColor, Me.SelectionWidth), Me.SelectionWidth \ 2, Me.SelectionWidth \ 2, ctl.Width - Me.SelectionWidth, ctl.Height - Me.SelectionWidth)
        End If

    End Sub

    Private Sub CreateCaret()
        insertCaret = New PictureBox
        With insertCaret
            .Name = "caret"
            .Height = 1
            .Width = CaretWidth
            .Margin = CaretPadding
            .Padding = New Padding(0)
            .BackColor = Me.CaretColor
        End With

    End Sub

End Class

The Minimum For Integration Logins

When you want to have your site support login from other services like Google, Facebook, or Microsoft, you use the Owin libraries.  The base MVC template sets all this up for you, but it also has a lot of stuff that your site probably already has, like login and a user database and whatnot.

So, what’s the minimum amount of code you need to add to handle the extra login sources.  You need two things: a class to initialize Owin and an MVC controller to handle the login redirect and the login response from the external site.

This is the class to initialize Owin:

Imports Microsoft.Owin
Imports Microsoft.Owin.Security
Imports Owin.Security.Providers

' Nuget Packages needed:
' Owin
' Microsoft.Owin
' Microsoft.Owin.Security
' Microsoft.Owin.Security.Coookies
' Microsoft.Owin.Host.SystemWeb
' Microsoft.Owin.Security.Facebook (for FB Login)
' Microsoft.Owin.Security.Google (for Google Login)
' Microsoft.Owin.Security.MicrosoftAccount (for MS Login)
' Owin.Security.Providers (for many other Logins)

<Assembly: Microsoft.Owin.OwinStartup(GetType(Startup))>
Public Class Startup
    Public Sub Configuration(app As Owin.IAppBuilder)
        Dim opt As Cookies.CookieAuthenticationOptions

        opt = New Cookies.CookieAuthenticationOptions With {.LoginPath = New PathString("/Account/Login")}

        Owin.CookieAuthenticationExtensions.UseCookieAuthentication(app, opt)
        AppBuilderSecurityExtensions.SetDefaultSignInAsAuthenticationType(app, opt.AuthenticationType)

        ' Google - Signup https://developers.google.com
        Owin.GoogleAuthenticationExtensions.UseGoogleAuthentication(app, New Google.GoogleOAuth2AuthenticationOptions With {.ClientId = "", .ClientSecret = ""})

        ' Facebook - Signup https://developers.facebook.com/
        Owin.FacebookAuthenticationExtensions.UseFacebookAuthentication(app, New Facebook.FacebookAuthenticationOptions With {.AppId = "", .AppSecret = ""})

        ' Microsoft - Signup https://account.live.com/developers/applications/index (RedirectURL in app settings must be http://domain.com/signin-microsoft) 
        Owin.MicrosoftAccountAuthenticationExtensions.UseMicrosoftAccountAuthentication(app, New MicrosoftAccount.MicrosoftAccountAuthenticationOptions With {.ClientId = "", .ClientSecret = ""})

        ' Yahoo - Signup http://developer.yahoo.com (not working; 401 errors) 
        Yahoo.YahooAuthenticationExtensions.UseYahooAuthentication(app, New Yahoo.YahooAuthenticationOptions With {.ConsumerKey = "", .ConsumerSecret = ""})

        ' Flickr - Signup https://www.flickr.com/services/api/keys/
        Flickr.FlickrAuthenticationExtensions.UseFlickrAuthentication(app, New Flickr.FlickrAuthenticationOptions With {.AppKey = "", .AppSecret = ""})

    End Sub

End Class

And this is the controller class to manage the logins:

Imports System.Web.Mvc

Namespace Controllers
    Public Class OwinController
        Inherits Controller

        Function Login(provider As String, returnURL As String) As ActionResult
            Return New ChallengeResult(provider, Url.Action("Callback", "Owin", New With {.ReturnURL = returnURL}))
        End Function

        Function Callback(returnURL As String)
            Dim userID As String
            Dim email As String

            ' capture credentials
            With DirectCast(My.User.CurrentPrincipal.Identity, Security.Claims.ClaimsIdentity)
                userID = .Claims.Where(Function(x) x.Type.EndsWith("/nameidentifier")).DefaultIfEmpty(New Security.Claims.Claim("", "")).First.Value
                email = .Claims.Where(Function(x) x.Type.EndsWith("/emailaddress")).DefaultIfEmpty(New Security.Claims.Claim("", "")).First.Value
            End With

            Debug.WriteLine(My.User.CurrentPrincipal.Identity.Name)

            Return New RedirectResult(returnURL)

        End Function

        Private Class ChallengeResult
            Inherits HttpUnauthorizedResult

            Public Property LoginProvider As String
            Public Property RedirectURL As String

            Public Sub New(provider As String, url As String)
                Me.LoginProvider = provider
                Me.RedirectURL = url
            End Sub

            Public Overrides Sub ExecuteResult(context As ControllerContext)
                Dim prop As New Microsoft.Owin.Security.AuthenticationProperties With {.RedirectUri = Me.RedirectURL}
                context.HttpContext.GetOwinContext.Authentication.Challenge(prop, Me.LoginProvider)
            End Sub

        End Class

    End Class

End Namespace

The Callback method in the OwinController is where you would look up the user by their provider/nameidentifier combination or store that info in an existing logged-in user profile or create a new user profile for the new login.

When you want to offer external logins, you call the Owin controller:

@Html.ActionLink("Google Login", "Login", "Owin", New With {.Provider = "Google", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Facebook Login", "Login", "Owin", New With {.Provider = "Facebook", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Yahoo Login", "Login", "Owin", New With {.Provider = "Yahoo", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Flickr Login", "Login", "Owin", New With {.Provider = "Flickr", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Microsoft Login", "Login", "Owin", New With {.Provider = "Microsoft", .ReturnURL = Url.Action("Secure")}, Nothing)

This assumes there is an action named “Secure” that you want to come back to.

Outlook Rules Utility

Way back in 2013, I was getting annoyed with the rules manager in Outlook.  See, I have many rules.  I have one folder for every person in my inbox.  The rules move their messages to their respective folders.  So at work, I have 67 rules.  At home, I have 267 rules.  I like to keep my rules sorted, so I can find one quickly if I need to.  So if I add a rule for Zillow.com, I have to click the “move down” arrow hundreds of times to get to the bottom of the list.  There has to be a better way.

I focused on the rules export, which creates a binary file of all your rules.  I just had to reverse-engineer the file structure and I could create an app to modify the order of the rules.  Over a span of time, I figured out what I needed and the results were somewhat workable.  I uploaded the source code to CodePlex and didn’t really use it much after that.

Two and a half years later, I was annoyed at my rules again and fired up this code to fix my rules.  I discovered the code didn’t work like I expected, so I set to work fixing the problems.  Once it was working properly, I decided that the CodePlex project deserved to have a binary for download.  It wasn’t doing a lot of good to people who weren’t programmers and who could compile the code on their own.

After that update, I did a quick search to see if anything new had been learned about the Outlook rules file since I last worked on it.  Nothing.  But what I did stumble on was a post asking how to parse the rules file.  In the post, the author explained where the rules file was stored in the mail store file.  Taking that information, I was able to add the ability for my utility to read and write the rules directly from Outlook.  No more export/import hassle!  That is a huge step forward.

This unexpected progress has inspired me to work a little harder on the project.  There are random bits of code and comments that handle the different criteria and the different actions.  These need to be clarified and once that is done, this utility could be an entire replacement for the Rules Manager in Outlook.

The project is at http://rwzreader.codeplex.com.

GrooveManager 1.0 Release

I’ve been posting for a little while on Groove Music’s database.  While investigating, I wrote up some test utilities to read and write to the Groove Music database.  These utilities have been combined and simplified into a utility called GrooveManager.

At this point, the utility is extremely limited.  It will only edit Artist names and the Artist image.  And the Artist image feature itself is limited in that you can’t use external images.  It has to be an artist with a profile on Xbox Music.  That limitation aside, there are still times when you have a legitimate need to use an artist that isn’t spelled the same as you have it in your collection.  For example, Sergei Rachmaninoff seems to have many different spellings and variations.

The project page is at https://groovemanager.codeplex.com and has simple documentation on the website.

Groove Metadata (database)

Here’s some more info on the structure of the Groove Music database.

Schema

As mentioned the database is in ESE format.  The tables within are pretty much just like any other database.  And in Groove Music’s case, you have the expected elements of Artist, Album, Tracks, and Genre.  Most of the fields are what you would expect them to be.  Here’s a brief analysis on some things you might be curious about.

tblPerson

This table holds the Artist info, but it’s more than just the band name.  It also includes credits in each song’s ID3 tag, like composer, artist, albumartist, etc.  This means you’re going to have a lot more “Persons” than bands in your library.  Especially when dealing with compilation albums.  Other fields:

tblAudioAlbum

This table is the list of albums in the library.

tblTrack

This table lists the songs.  There’s lots of interesting info in here that needs more exploration.  I have three albums in the cloud.  Two I purchased back from the Zune days, and one I just uploaded to OneDrive from my hard drive.  I’ll reference these in the interesting fields.

  • CloudCollectionContentId – This seems to be a guid to use in a link to an online file.  This is populated for both my purchased and uploaded albums
  • BlockFromCloud – Always true
  • ActionableMediaId – I assume this is a Guid that points to a matched online track. It’s blank on my more obscure albums.
  • InCloudCollection – True for the three albums I have in OneDrive and Store-purchased
  • CatalogId – The same as ActionableMediaId, but there are more CatalogIds than ActionableMediaIds.  Maybe ActionableMediaId means it can be used online instead of just having information?
  • AlbumImageId – A Guid that is used to get the album art from web services
  • AlbumImageSource – A number that likely represents the URL to get the image from (zuneimages, music.microsoft.com, or musicimages.xbox.com).
  • UniqueFileId – These values are similar to what Windows Media Player used to store in the ID3 tag when it would apply its metadata from its store.

So still, plenty to learn about this database.