The first Kata is about a division by zero exception that happens on startup, specifically a .net 4.5 console application (that’s explicitly set to x64 instead of AnyCPU and debugged with a 64-Bit WinDbg just so that I have a consistent environment) whose main method is this:
static void Main(string[] args)
{
// To make sure the Compiler doesn't catch or optimize this
int x = 10 - 10;
// This blows up with a Division By Zero exception
int i = 2000 / x;
Console.WriteLine("The result of 2000 / {0} is {1}.", x, i);
}
Running this application results in an immediate crash, with no chance to attach a debugger. We start out by opening WinDbg and File > Open Executable. This results in WinDbg starting the app and immediately breaking into the debugger.
CommandLine: F:\WinDbgKatas\001-DivByZeroOnStartup\001-DivByZeroOnStartup\bin\Release\001-DivByZeroOnStartup.exe
Symbol search path is: SRV*C:\Users\mstum\Desktop\MSDN\DEBUG\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
ModLoad: 00000001`3fb50000 00000001`3fb56000 001-DivByZeroOnStartup.exe
ModLoad: 00000000`77290000 00000000`77439000 ntdll.dll
ModLoad: 000007fe`f86f0000 000007fe`f875f000 C:\windows\SYSTEM32\MSCOREE.DLL
ModLoad: 00000000`77170000 00000000`7728f000 C:\windows\system32\KERNEL32.dll
ModLoad: 000007fe`fd3d0000 000007fe`fd43c000 C:\windows\system32\KERNELBASE.dll
(30c4.337c): Break instruction exception - code 80000003 (first chance)
ntdll!LdrpDoDebuggerBreak+0x30:
00000000`7733cb60 cc int 3
Debug > Go (or the g command line) makes it run and crash.
0:000> g
ModLoad: 000007fe`ff4c0000 000007fe`ff59b000 C:\windows\system32\ADVAPI32.dll
ModLoad: 000007fe`fe470000 000007fe`fe50f000 C:\windows\system32\msvcrt.dll
ModLoad: 000007fe`fd910000 000007fe`fd92f000 C:\windows\SYSTEM32\sechost.dll
ModLoad: 000007fe`fdf40000 000007fe`fe06d000 C:\windows\system32\RPCRT4.dll
ModLoad: 000007fe`f8650000 000007fe`f86e9000 C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscoreei.dll
ModLoad: 000007fe`fda80000 000007fe`fdaf1000 C:\windows\system32\SHLWAPI.dll
ModLoad: 000007fe`fd930000 000007fe`fd997000 C:\windows\system32\GDI32.dll
ModLoad: 00000000`77070000 00000000`7716a000 C:\windows\system32\USER32.dll
ModLoad: 000007fe`fdd50000 000007fe`fdd5e000 C:\windows\system32\LPK.dll
ModLoad: 000007fe`fd490000 000007fe`fd559000 C:\windows\system32\USP10.dll
ModLoad: 000007fe`fd610000 000007fe`fd63e000 C:\windows\system32\IMM32.DLL
ModLoad: 000007fe`fdba0000 000007fe`fdca9000 C:\windows\system32\MSCTF.dll
ModLoad: 000007fe`f7c50000 000007fe`f85ae000 C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
ModLoad: 000007fe`f7b70000 000007fe`f7c42000 C:\windows\system32\MSVCR110_CLR0400.dll
(30c4.337c): Unknown exception - code 04242420 (first chance)
ModLoad: 000007fe`f65f0000 000007fe`f7b6d000 C:\windows\assembly\NativeImages_v4.0.30319_64\mscorlib\1b61dcfd88eae971a10423d914e1014a\mscorlib.ni.dll
ModLoad: 000007fe`ff2b0000 000007fe`ff4b3000 C:\windows\system32\ole32.dll
ModLoad: 000007fe`fcef0000 000007fe`fceff000 C:\windows\system32\CRYPTBASE.dll
ModLoad: 000007fe`f64c0000 000007fe`f65ee000 C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clrjit.dll
ModLoad: 000007fe`fd9a0000 000007fe`fda77000 C:\windows\system32\OLEAUT32.dll
(30c4.337c): Integer divide-by-zero - code c0000094 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
From here, we can load the .net debugging extensions through .loadby sos clr (remember, in .net 4.0 it's clr instead of mscorwks) Now, this is a first chance exception and as the text says, it’s reported before any exception handling. Specifically, it’s not on the managed stack yet, so !pe won’t find it:
0:000> .loadby sos clr
0:000> !pe
There is no current managed exception on this thread
At this point, we can look at the !ClrStack to get an idea where we crashed.
0:000> !ClrStack
OS Thread Id: 0x337c (0)
Child SP IP Call Site
000000000049ea60 000007fe9860009f *** WARNING: Unable to verify checksum for 001-DivByZeroOnStartup.exe
WinDbgKata.DivByZeroOnStartup.Program.Main(System.String[]) [f:\WinDbgKatas\001-DivByZeroOnStartup\001-DivByZeroOnStartup\Program.cs @ 12]
000000000049ed90 000007fef7c9f713 [GCFrame: 000000000049ed90]
Entering g again will give us a second chance, this time the Exception is on the managed stack.
0:000> g
ModLoad: 000007fe`fc140000 000007fe`fc14c000 C:\windows\system32\VERSION.dll
ModLoad: 000007fe`f3f70000 000007fe`f4079000 C:\Windows\Microsoft.NET\Framework64\v4.0.30319\diasymreader.dll
(30c4.337c): Integer divide-by-zero - code c0000094 (!!! second chance !!!)
KERNELBASE!RaiseException+0x39:
000007fe`fd3dbccd 4881c4c8000000 add rsp,0C8h
0:000> !pe
Exception object: 0000000002572ed8
Exception type: System.DivideByZeroException
Message: Attempted to divide by zero.
InnerException: <none>
StackTrace (generated):
SP IP Function
000000000049EA60 000007FE9860009F 001_DivByZeroOnStartup!WinDbgKata.DivByZeroOnStartup.Program.Main(System.String[])+0xf
StackTraceString: <none>
HResult: 80020012
Depending on the exact exception we can now dig further, since it's on the managed stack we can !do it. This won't give us much more information in the Division by Zero case, but if there's an InnerException
0:000> !do 0000000002572ed8
Name: System.DivideByZeroException
MethodTable: 000007fef6c6f020
EEClass: 000007fef675f068
Size: 160(0xa0) bytes
File: C:\windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fef6c8aee0 4000002 8 System.String 0 instance 000000000257ae30 _className
000007fef6c89460 4000003 10 ...ection.MethodBase 0 instance 0000000000000000 _exceptionMethod
000007fef6c8aee0 4000004 18 System.String 0 instance 0000000000000000 _exceptionMethodString
000007fef6c8aee0 4000005 20 System.String 0 instance 0000000002578fd0 _message
000007fef6c88ee8 4000006 28 ...tions.IDictionary 0 instance 0000000000000000 _data
000007fef6c8b110 4000007 30 System.Exception 0 instance 0000000000000000 _innerException
000007fef6c8aee0 4000008 38 System.String 0 instance 0000000000000000 _helpURL
000007fef6c8b4c0 4000009 40 System.Object 0 instance 0000000002579100 _stackTrace
000007fef6c8b4c0 400000a 48 System.Object 0 instance 0000000002579148 _watsonBuckets
000007fef6c8aee0 400000b 50 System.String 0 instance 0000000000000000 _stackTraceString
000007fef6c8aee0 400000c 58 System.String 0 instance 0000000000000000 _remoteStackTraceString
000007fef6c8dc90 400000d 88 System.Int32 1 instance 0 _remoteStackIndex
000007fef6c8b4c0 400000e 60 System.Object 0 instance 0000000000000000 _dynamicMethods
000007fef6c8dc90 400000f 8c System.Int32 1 instance -2147352558 _HResult
000007fef6c8aee0 4000010 68 System.String 0 instance 0000000000000000 _source
000007fef6c8ed00 4000011 78 System.IntPtr 1 instance 0 _xptrs
000007fef6c8dc90 4000012 90 System.Int32 1 instance -532462766 _xcode
000007fef6c77828 4000013 80 System.UIntPtr 1 instance 0 _ipForWatsonBuckets
000007fef6cf2960 4000014 70 ...ializationManager 0 instance 00000000025790c8 _safeSerializationManager
000007fef6c8b4c0 4000001 0 System.Object 0 shared static s_EDILock
>> Domain:Value 00000000005c4a50:NotInit <<
May 16th, 2013 in
Development | tags:
kata,
windbg |
No Comments
One of the most popular browser extensions is AdBlock, which prevents loading of ads in websites. Now, there isn’t much wrong with ads itself, except that all the ad networks like to spy on you with tracking cookies and more malicious methods like supercookies that use browser plugins. There are some ridiculous attempts at allowing users to opt out of that (e.g., the complete failure that Do-Not-Track is) but these will never work because ad networks will continue to try everything they can to play dirty.
AdBlock browser extensions are one way to play dirty in return, but it’s not that effective: It only protects one browser, but won’t protect your smartphones, tablets or other machines without adblock on your network. You need to go deeper: You need to block malicious sites in your router, so that each and every device that goes through your router. Specifically, you want to override DNS entries to set malicious domains to 0.0.0.0. Any decent Linux-Firmware should do that, in my case I run TomatoUSB on my Asus RT-N16 router. (Disclaimer: Flashing new Firmware is done at your own risk. If you don’t understand the instructions on the Tomato Website, get someone who does 
You will need to properly setup the whole thing, including your network configuration, DHCP, WiFi with security etc.).
Under Administration > Scripts > WAN Up, I have this script:
## Adblock script [Version 2.1 | 08 July 2008 | 3778 bytes]
##
## Created by Adrian Jon Kriel: root-AT-extremecooling-DOT-org
## Modified
## tomato WAN Up script
##
## 0 = disable
## 1 = enable
## (1) = default value
## optimising of dnsmasq, (1)
eval OPTDNSMASQ="1"
## automatic updating, (1)
eval AUTOUPDATE="1"
## MVPS HOSTS ~18,500 lines, 680 Kbyte, (1)
eval MVPSSOURCE="1"
## pgl.yoyo.org ~2,200 lines, 68 Kbyte, (1)
eval PGLSOURCE="1"
## hosts-file.net ~53,000 lines, 1.5 Mbyte, (0)
eval HSFSOURCE="0"
## Hosts File Project ~102,000 lines, 3.0 Mbyte ***6+mb free memory***, (0)
## This actually makes the RT-N16 grind to a halt, I guess it's not powerful enough.
eval HFPSOURCE="0"
##
## varibles
## location of temp file, (/tmp/hosts)
eval GENFILE="/tmp/hosts"
## redirect ip, (0.0.0.0)
eval REDIRECTIP="0.0.0.0"
## sources
eval MVPSOURCEFILE="http://www.mvps.org/winhelp2002/hosts.txt"
eval PGLSOURCEFILE="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts;showintro=0"
eval HSFSOURCEFILE="http://support.it-mate.co.uk/downloads/hosts.txt"
eval HFPSOURCEFILE="http://hostsfile.mine.nu/Hosts"
if ping -c 1 yahoo.com ; then
eval GOTSOURCE="0"
echo "" > $GENFILE
## download
if [ "$MVPSSOURCE" = "1" ] ; then
if wget $MVPSOURCEFILE -O - >> $GENFILE ; then
logger ADBLOCK Downloaded $MVPSOURCEFILE
eval GOTSOURCE="1"
else
logger ADBLOCK Failed $MVPSOURCEFILE
fi
fi
if [ "$PGLSOURCE" = "1" ] ; then
if wget $PGLSOURCEFILE -O - >> $GENFILE ; then
logger ADBLOCK Load $PGLSOURCEFILE
eval GOTSOURCE="1"
else
logger ADBLOCK Fail $PGLSOURCEFILE
fi
fi
if [ "$HSFSOURCE" = "1" ] ; then
if wget $HSFSOURCEFILE -O - >> $GENFILE ; then
logger ADBLOCK load $HSFSOURCEFILE
eval GOTSOURCE="1"
else
logger ADBLOCK Fail $HSFSOURCEFILE
fi
fi
if [ "$HFPSOURCE" = "1" ] ; then
if wget $HFPSOURCEFILE -O - >> $GENFILE ; then
logger ADBLOCK Load $HFPSOURCEFILE
eval GOTSOURCE="1"
else
logger ADBLOCK Fail $HFPSOURCEFILE
fi
fi
if [ "$GOTSOURCE" = "1" ]; then
logger ADBLOCK Got Source Files
#FREE MEMORY!
service dnsmasq stop
killall -9 dnsmasq
logger ADBLOCK Ignor Fail Safe
##strip source file
sed -i -e '/^[0-9A-Za-z]/!d' $GENFILE
sed -i -e '/%/d' $GENFILE
sed -i -e 's/[[:cntrl:][:blank:]]//g' $GENFILE
sed -i -e 's/^[ \t]*//;s/[ \t]*$//' $GENFILE
## dnsmasq, sanitize, optimised
sed -i -e 's/[[:space:]]*\[.*$//' $GENFILE
sed -i -e 's/[[:space:]]*\].*$//' $GENFILE
sed -i -e '/[[:space:]]*#.*$/ s/[[:space:]]*#.*$//' $GENFILE
sed -i -e '/^$/d' $GENFILE
sed -i -e '/127.0.0.1/ s/127.0.0.1//' $GENFILE
sed -i -e '/^www[0-9]./ s/^www[0-9].//' $GENFILE
sed -i -e '/^www./ s/^www.//' $GENFILE
## remove duplicates (resource friendly)
cat $GENFILE | sort -u > $GENFILE.new
mv $GENFILE.new $GENFILE
## format
sed -i -e 's|$|/'$REDIRECTIP'|' $GENFILE
sed -i -e 's|^|address=/|' $GENFILE
## load values from dnsmasq config
cat /etc/dnsmasq.conf >> $GENFILE
## optimise dnsmasq
if [ "$OPTDNSMASQ" = "1" ] ; then
cat >> $GENFILE <<EOF
cache-size=2048
log-async=5
EOF
fi
## remove/whitelist websites
sed -i -e '/ark.intel.com/d' $GENFILE
## apply blacklist
dnsmasq --conf-file=$GENFILE
## failsafe added
dnsmasq
logger ADBLOCK Ignor Fail Safe
## dev info
logger ADBLOCK Unique Hosts Blocked $(awk 'END { print NR }' $GENFILE)
else
logger ADBLOCK Error Not Downloaded
fi
else
logger ADBLOCK Error No Internet
fi
## remove the generated files
rm $GENFILE*
## automatic update
if [ "$AUTOUPDATE" = "1" ] ; then
## script exists
if [ -x /tmp/script_wanup.sh ] ; then
cru a UpdateAdlist "0 6,12,18,0 * * * /tmp/script_wanup.sh >/dev/null 2>&1"
fi
fi
## the end
Note that some of the sources no longer work, but it’s a good starting point. Some sites you may want to visit may be blocked, see the “remove/whitelist websites” part and add the ones you want to unblock.
Under Advanced > DHCP / DNS, make sure that “Use internal DNS” is ticked and add additional sites like this:
address=/googlesyndication.com/0.0.0.0
address=/google-analytics.com/0.0.0.0
address=/googleadservices.com/0.0.0.0
address=/doubleclick.net/0.0.0.0
address=/finder.cox.net/0.0.0.0
The Status > Web Usage page is extremely helpful to find additional hosts that need to be blocked - there is a war out there and the bad guys keep moving, so as a target we also need to stay vigilant and move with them.
I’m still trying to figure out if there is a way to do Layer 7 filtering, to only disallow certain subdirectories on a URL (e.g., prevent facebook.com/plugins without blocking Facebook as a whole) or to inspect and block or modify requests (e.g., add the (mostly) useless DNT: 1 header to each and every outgoing HTTP request, just in case. Or change cookie values/ids for "Social Media Share" plugins to random values.). Any decent Firewall does that (Stateful Packet Inspection), but I might want to get a more powerful router since the low memory and CPU speed (128 MB/480 MHz) on the RT-N16 makes some of the more interesting things slow down things a lot.
I have a machine that runs .net 4.0 and where I took a memory dump of an application. I moved the dump to my machine, which has .net 4.5 installed and tried to debug it in WinDbg:
0:000> .loadby sos clr
0:000> !DumpHeap
Failed to load data access DLL, 0x80004005
Verify that 1) you have a recent build of the debugger (6.2.14 or newer)
2) the file mscordacwks.dll that matches your version of clr.dll is
in the version directory or on the symbol path
3) or, if you are debugging a dump file, verify that the file
mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.
4) you are debugging on supported cross platform architecture as
the dump file. For example, an ARM dump file must be debugged
on an X86 or an ARM machine; an AMD64 dump file must be
debugged on an AMD64 machine.
You can also run the debugger command .cordll to control the debugger's
load of mscordacwks.dll. .cordll -ve -u -l will do a verbose reload.
If that succeeds, the SOS command should work on retry.
If you are debugging a minidump, you need to make sure that your executable
path is pointing to clr.dll as well.
On the internet, there’s a bunch of guides that focus on bringing in the right version of mscordacwks.dll, but that wasn’t my problem (I have setup the Symbol Server and WinDBG was correctly downloading the right version). Turns out that .net 4.5 comes with a new SOS.dll that is incompatible with .net 4.0 memory dumps. The solution for me was to copy the SOS.dll from a .net 4.0 machine (from C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319) into the winext folder of my WinDbg installation (C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x86\winext) and then just load it with
.load sos
Yeah, this whole in-place upgrade of .net 4.0 to 4.5 was truly a great idea…
May 13th, 2013 in
Development | tags:
.net,
sos,
windbg |
1 Comment
Yeah, I’m late to the party, sue me if you want 
(I’m also late with my Indiana Jones stuff, I know). Anyway, I’ve just finished implementing OAuth on both a server and client, using .net for an additional challenge,
OAuth 1.0a is a decent protocol. One of the things it gets right is also the one biggest problem: The weird parameter reordering to create the HMAC-SHA1 signature. From what I’ve read, this was because the PHP installation on a certain webhost didn’t give access to the raw QueryString, hence this mechanism was created.
It’s the single biggest issue, but it also shows how the web works: There is a huge web hoster that has a limitation. The purist in me wants to say “Screw that Host” but at the end, you want adoption, and that means catering to the lowest common denominator. On the other hand, the OAuth guys kept HMAC-SHA1 as a security mechanism.
If you get this part right, OAuth 1.0a is a simple and elegant protocol. Really, it’s awesome. It has one downside that is a problem on mobile apps: It expects you to hit a service through HTTP and wants a callback URL. Some Services (e.g., Twitter some time ago) allowed you to hit their OAuth endpoint and get a numeric code that you could use to verify the request (which is neat for desktop apps), but it’s not really standardized.
The other problem: Access Tokens in OAuth 1.0a aren’t meant to expire, and you can’t really use it from JavaScript. If you want JS, the best solution is to hit your own backend which makes the OAuth request. And expect Access Tokens to expire.
OAuth 2 was supposed to fix this, but oh boy, this protocol is such a heap of garbage I don’t even know where to start. Luckily, one of the guys working on the Spec wrote it better than I could do.
If you are working on a Service Provider – either on the Internet or an Intranet – take the time to read up on OAuth and don’t make the mistake to discount OAuth 1.0a as “old stuff” and 2.0 as the new hotness. OAuth 2.0 offers some improvements for mobile phones, but it makes all security optional and has so many implementation details and additional complexities, implementing it may be a pain (also, some services are still on old versions of it).
Seriously take a look at OAuth 1.0a. If you’re on .net, look at DotNetOpenAuth which does a good job implementing it for servers and clients alike. There is not much documentation, but download the source and look at both the samples and the source. Also, use Fiddler to troubleshoot. The beauty of HTTP is that we can troubleshoot stuff super easily.
There are still a lot of headaches around testing, but if you find yourself in a situation where you need to solve the NTLM double hop problem or other similar situations where you have clients you can’t trust, OAuth 1.0a is a good protocol to look into.
RFC 5849 standardizes it (the RFC says OAuth 1.0, but it’s 1.0a).
April 17th, 2013 in
Development | tags:
oauth |
No Comments
In 1997, there was a game that was not simply great but truly genre-shaping - Castlevania: Symphony of the Night. It took Castlevania but turned it into a deep, open world action RPG not unlike Metroid but with it’s own unique RPG elements. Thus, a new genre was created: Metroidvania. Games in which you had a broad world to explore with many, many hidden rooms that require you to gain powers and then backtrack, all to be rewarded with secrets (in case of SOTN, they literally hid half of the game behind one such secret).
A few years back in 2009, my personal favorite game in the genre was released, the brilliant Shadow Complex. Seriously, if you own an XBox and you are a gamer, you owe it to yourself to buy it if you haven’t done so.
Today, another Metroidvania game was released, Guacamelee! I’m playing it on my Vita right now. The game is heavily nodding towards Mexican culture, specifically the Dia de muertos (Day of the Dead) and Mexican wrestling. It looks very stylized and has you play as a reborn luchador who has to rescue the daughter (and your childhood friend) of El Presidente.
The setting is a lot more lighthearted than Metroid, Castlevania or Shadow Empire, with special moves like “Downercut” or “Rooster Uppercut” and the combat is based on melee-combos rather than weapons (hence the name, in itself a light-hearted pun on Guacamole). It does a great job to tease you with locked passages (red, green, blue and white blocks that require a special ability to pass) and secrets (e.g., heart containers) which already makes me excited about backtracking and unlocking these passages (that is one of my favorite things about Zelda games).
The music is good, definitely with the Mexican flair you’d expect (yes, that includes trumpets). It’s available on Playstation 3 and Vita for $15 (or $12 if you’re a PS Plus subscriber) and it’s cross-buy, so one purchase will give you both platforms. It also supports cross-save through the PSN cloud, which means that you can play on both platforms on the save savegame. This is super neat!
I haven’t finished the game yet, so I don’t know how it holds up compared to Shadow Complex, but if you like Metroidvania games, Guacamelee! is worth it.
When someone asks me about my opinion about Microsoft, my answer will be among the lines of this: “Their Development Tools are a shining beacon on a sea of darkness, they have moved into the future at a time when everyone was still trying to polish the crap from the 1970’s. But instead of ruling the tech world with high quality products, the inability of their management to execute has allowed other companies to dominate them”. Today, I want to look at one of the many technologies that meet that description.
LINQ to SQL (or L2S) is a phenomenal technology and as of February 2013 still my ORM of choice. But the amount of failure around this product is stunning. First of, L2S is an ORM made for a single database: Microsoft SQL Server. It can’t do Oracle. It can’t do MySQL. And it won’t do any of the shiny new NoSQL databases. It does do at least three versions of SQL Server (2005/08/10, haven’t tried 2000 or 2012) though, and it does them extremely well.
There are several complaints about it though, one of them being the SQL that it generates. After trying out other ORMs, I found that L2S generates by far the best SQL. Open up System.Data.Linq in dotPeek and look at the complexity within it. I don’t know how many manhours went into this, but all of this was made to generate good SQL, because it was focused on a single underlying database. Unless EF significantly improved over the last 6 months, it is nowhere close to generate good SQL. (Side note: Entity Framework didn’t support mapping columns to enums for a long time. I think they added that, but this was the thing that told me that EF is not a serious product.)
Sometimes, your queries cause joins or subselects that you did not expect. Accidentally writing a foreach-loop that makes 1 call per iteration (rather than 1 bigger call before the loop which is iterated over) is easy, in fact it’s almost the default in a lot of cases. Finding these instances is hard because your only built-in tool is the SQL Server Profiler, and that one doesn’t come with SQL Express. I bought Ayende’s L2SProf and I am still almost recommending it to everyone working with it, it is THE #1 debugging tool. Why did I write “almost” then? Because Sam Saffron in his infinite awesomeness made MiniProfiler, which is not only a requirement for any serious ASP.net MVC developer, but it does a spectacular job capturing and analyzing the generated SQL.
Once you are past the stupid but super-common SELECT N+1 problem, you can tweak the LoadOptions to force joins/subselects (e.g., “I know that when I select a ParentObject, I need its ChildObjects as well, so let’s tell L2S to fetch them automatically, thus avoiding or at least improving on SELECT N+1”).
That last point (Monitoring Queries) is a good Segway into another issue, the tooling support. Microsoft loves Designer Surfaces, and so it is no wonder that L2S is driven by one as well. To work with L2S you need to generate database classes, and this is usually done though a .dbml file. This is one of the weakest points, because that stuff is awful:
- If you want to map a column to an enum, you often need to prefix the enum type with global:: (e.g., global::MyApplication.Models.MyEnum). If you don’t you get an unhelpful error message.
- If you ever change the table definition in the database, there is no “Refresh Table from DB” command. You either manually need to add the column or remove and readd the table on the design surface, losing all other changes (like enum mappings).
- The Code was generated by a tool, and it looks like it. It’s ugly.
DBML files are garbage. So I went ahead and created a T4 template that connects to the database, reads the table definitions and creates L2S Classes. If I change the schema, one click on “Run Custom Tool” regenerates the whole lot. The generated code is green in Resharper. Classes can be partial if I need to extend them without affecting the regeneration. Classes implement ICloneable and a CompareWithClone method that allows me to get a list of changes between two instances (e.g., for Auditing purposes – Fetch record, clone, modify record, compare clone with modified record) and it does so without Reflection. Foreign Key relations? No problem, EntityRef/EntitySet is generated. Writing this made my experience with L2S a million times better. I do not have it to share at the moment though (working on that).
The future of L2S is in doubt since that “Update on LINQ to SQL and LINQ to Entities Roadmap” came out. However, it got a massive update in .net 4.0, despite concerns. I think it even got support for SPARSE columns in a later update.
It has a set of issues (e.g., the mapping code is SLOW) and it seems unlikely that these issues are resolved. And this is where Microsoft’s inability to execute properly shines through. Integrating with SQL Server is a beauty in .net, much better than working with the awful, awful Oracle or MySQL client libraries. It is not as sexy as the NoSQL stuff (where RavenDB is the product of choice if you want a document db) but SQL Server is the workhorse that does 95+% of the work in all applications and it’s the default choice for .net.
I don’t want a product that gives me half-assed integration with multiple databases, because I don’t need to support more than one, and chances are high you don’t need either. I want a single kick ass technology for SQL Server. I don’t want to write mundane CRUD SQL queries manually, because these aren’t the 1970’s anymore. But when I do notice a bottleneck, I want to be able to optimize it. L2S does that (did you know that DataContext.ExecuteCommand allows execution of SQL?). I love Dapper, but it’s a bit too much plumbing for the simple, common stuff, sorry. I want to reduce complexity and code size, not increase it by hand-optimizing a simple SELECT. That’s like writing a game fully in x86-64 assembly language instead of using a more high level language and only go down to assembly where it offers an advantage worthy of the price.
As an external dev, I can fix some of it. I fixed the DBML madness. I fixed the testing issues (see that generated XML file? It contains enough data for me to spin up either a SQL User instance or a SQL Compact database, recreate the schema and then clean up at the end of the test).
But I can’t fix the slow mapping and I can’t add any support for SQL Server 2012 improvements. And if Microsoft doesn’t want to, this is just another slightly unpolished jewel that they are throwing away instead of making it shine.
I do not know who owns LINQ to SQL internally at Microsoft, but here is my plea: Please open source LINQ to SQL. You have done so with a lot of the ASP.net stuff and in my opinion it made it a better product. I don’t want to decompile the DLL and try making my own, illegal fork of it. I want to go in and fix the performance issues around mapping. I want to sit down with a database guy and the debugging tools to see where queries can be optimized.
I want LINQ to SQL to be the best tool to work with the sanest RDBMS choice on .net and even though it already holds that spot, I want to see it leave the competition in the dust. Please Microsoft, stop screwing this stuff up.
Thanks,
A developer that uses the heck out of L2S every single day
(Note: This is a work of fiction that should normally go on Dime Fiction, except that I’m closing that blog because my inspiration comes in waves and isn’t focused enough yet...)
Nurob was woken in the morning by a strange noise. Strange noises weren’t uncommon in his line of work, but today they were especially strange. He quickly got dressed and spoke to Nahira, his best foreman. She was working for the engine since several years, but what made her stand out was her ability to analyze even the most subtle of issues.
He remembered when one of this engineers would just look at a slight deviation from the norm as a “minor malfunction of the sensors”, but later on it would actually cause a major Hurricane on some remote planet, and he remembered the locals calling it “Sandy” for whatever strange reason.
To be fair, that “remote planet” was more important to them than anything else. it was the one planet in this system that currently bore life. Nurob remembered a careless predecessor in his job that ignored warning signals which turned out to be genuine, and it cost them a whole planet of life.
As Engine Guardians, they were technically not responsible for anything out of the Norm, but losing the planet that was commonly called “Mars” caused a big investigation from the Creators and eventually led to Nurob’s appointment to guardian. So he knew that the last remaining planet with life was sacred to him. The creators were reasonable people that liked to experiment, but at the same time they were also serious about what they were doing. They knew that creating life was non-trivial, and that creating sentience was even harder. Wasting sentient species for no good reason was pretty much the only reason why he got his job, and Nurob had no intention or losing it.
It amused and concerned him to equal parts how Nahira was treated by her peers. Their species did not really differentiate between Males and Females, but there were subtle differences, and the Creators made a point of splitting any species into two halves, but all of the Males discounted Nahiras observations as simple female intuition that was as misguided as the human concept of Religion. But Nurob learned to trust Nahira a long time ago when she predicted a giant impact that almost lost them the last remaining planet.
He quickly dressed in simple pants and robes to avoid any accusation of indecency and ran down the hallway. On his way, he looked at the giant dials that were showing details like temperature, rotation speed, solar power output and other less important numbers. These numbers looked good at first glance, but Nurob knew that something, somewhere must have been off. As he entered the monitoring chamber, Nerakul took notice.
“Nurob! Why are you here? Your shift doesn’t start yet!” he asked, but Nurob was too concerned to answer in a meaningful way. “Where is Nahira? I need to see her immediately because she sounded the alarm!” Nerakul was visibly disgusted. “Do you really believe in that nonsense? You saw the numbers on your way. The Jewel of Life is getting supplied with enough solar energy to sustain for several thousand years! Nahira is just having one of her intuitions, you know how women are!”
Nurob ignored him. He was used to Nerakul’s dismissive attitude and he knew that it was mostly justified, but the risk was too high. Everything related to that strange planet the locals called “Earth” was top priority because a loss would be catastrophic. He finally found Nahira in a corner, monitoring some solar flare activity. “Nahira, you rang the alarm! What is wrong?” She was visibly nervous, and Nurob was wondering if it was because of him or because of the situation at hand. He had some time thinking about her, so he knew that she only just started her job a few cycles ago and never had any special occasions to deal with. “Sir, I saw that the Solar Energy output dipped by 16% to only 1012.24 units. What made me concerned is that the engine temperature also dropped by 17 degrees. I fear that we may not get the Solar Energy up again within the next three or four cycles.”. He listened carefully and took a moment to think. The last time the Solar Energy dropped by more than 10 percent, the planet they all cared so much about went through something called an “Ice Age”. Nurob was only a simple worker back then, a Grunt that kept the machine running. But he remembered the big commotion that was going on at the time. People were panicking about losing the planet, even though everyone knew that a lot of mammals would easily survive. Nurob could not let this happen.
After going through some more diagnostic checks that were available in the monitoring chamber that didn’t lead to any meaningful results, he knew that he would have to go where none of the guardians went in a long time. He had to look at the core of the Engine. To a primitive mammal from the planet Earth, this might have been extremely impressive. But the Guardians knew that the Engine was just an extremely old design that was kept alive through good maintenance and sheer luck. He noticed Nahira on the other side of the corridor. “Nahira!” he shouted “I could use a hand to fix this. Since you’re the only Guardian who seems to notice this stuff you’re also the only one I trust to help me”. She was visibly impressed and hesitated for a moment, then nodded and walked towards him. “Grand Guardian, I have no experience with the Engine itself, are you really sure?” He just looked at her and waved his arm towards the hatch. “You have as much experience as I do, which is to say, not very much. But fools like Nerakul don’t notice small differences, so how would they be able to adjust an Engine of this size?”
The Engine was indeed massive. Built billions of cycles ago, it was made with ancient technology that required several hundreds of components for even the simplest tasks, but it also had a certain nostalgic beauty to it. Nurob had worked on much more modern engines, but he always felt that the new stuff lacked personality, or as he called it, “soul”, a word he learned during his studies of the planet he was appointed to guard. This old engine was a piece of art, build by the creators when they had their Bipedal phase of creation. Duality was a recurring theme: Two Legs, Two Arms, Two Eyes, Two Genders, Two Personalities. Over time, they experimented a bit more with different numbers, and while some experiments were successes (increasing Gender diversity worked really well in more civilized areas of the world) there were also a lot of failures (increasing the number of legs to four created some truly useful species, but going past that only caused creatures everyone universally hated). Nurob often wondered why the two personalities were allowed to exist (he personally preferred the personality that loved life and embraced fun over the more protective one), but the Creators made it clear that any disturbance would have severe implication to the Guardians, so he obeyed.
They spent several hours investigating the mechanical part of the Engine, but everything seemed in order. It was an elegant (if somewhat verbose) design and thus easy to troubleshoot, so they moved over to the software side of things. Nurob often admired the humans for their naiveté, believing in omniscience of their Creators even though they are really just trying out a lot of genes to see which work and then leave the hard work to the Guardians. At the same time, being in the heart of the Engine was something no human would ever accomplish. Nahira had to smile when she logged in to the Administration Interface. Primitive races are easy to impress, and what could be more impressive than the power of light? It took only a little Energy to light up the sphere they were in, but Humans were impressed by what they called the “Sun”. And that was only the light! Nurob often wondered what would happen of Humans would see the real power behind Solar Energy, the giant Engine? He had witnessed the discovery of electricity, the realization that the Earth was round, the first Lunar Landing and the end of the Curse of the Black Sox. But as belief-shattering these events were, they couldn’t compare with the realization that there was a giant engine at the core of the most important celestial body, build by sentient beings and used for experimentation.
But as with any experiment, the Software made by the creators had flaws. Sometimes, it would allocate resources in the wrong places. It would do that a lot, actually. The common case would just result in slightly larger bones, but there were uncommon cases where a pregnant woman would give birth to two children that were joined in one body structure. At first, this was seen as a major issue, but upon closer investigation it became clear that the solution won’t be easy. There was an issue if multiple births happen at the same time, and the solution would cause a potential slowdown in births. As the earth population grew and grew, the issue would occur more and more often, but the Creators did not want to reduce the birth rate out of fear of the planet. Instead, they decided to intervene. They decided to inject knowledge about a workaround into the fetus of a man later known as Ben Carson and were delighted when he later on managed to resolve the issue after it happened to people. Of course, this workaround wasn’t viable for the general case and had some risks on its own, but compared to the risk of decreasing human birth rate, it was deemed acceptable.
Nahira saw the issue first. “It seems that the Creators didn’t expect this race to live past last year, or at least the past earth year” she said. “It looks like there is another Ice Age planned, although the documentation says that the next one should be pretty far off.” she finished before Nurob took a look at the data. Indeed, the Earth was due for a Reset, which meant hefty storms followed by cold temperatures for thousands of years. But that seemed inappropriate at the time. Those Humans had proven willingness to improve. Several times in the past, Nurob considered pushing the reset button, but someone somewhere always evolved one step further and saved the race. And even though there were many things he didn’t like, the current humans seemed a lot more evolved than any of their predecessors. In fact, they even started to treat their breeding gender as equals, something which the Guardians still struggled with.
They discussed several alternative solutions. Slowing down the Earth’s rotation around the Sun was the most promising solution, but the humans would very likely notice it and react like any other animal whose cage was disturbed – panic would spread and potentially cause irreparable damage. Moving the Engine closer to the Earth would have similar ripple effects to other planets and would be equally visible. Ever since a guy with a Mohawk shot a robot 300 million miles to space to land in a target area the size of a sheet of paper, the Guardians knew that the humans would notice every small change. “I remember writing code like this in school” Nahira eventually said. What concerned Nurob was the thing she hadn’t said. It was a high risk, this codebase was written a long time ago, and none of the original Creators were available for guidance. There was no test environment and no easy way to rollback any changes. There was no documentation as to why some of the choices were made. But there was also no way to add more duct tape to it. (Nurob once heard that expression from human engineers. He didn’t know what Duct Tape was, but the humans were so excited about it, Nurob would use the phrase a lot.)
Nahira sat down in front of the archaic terminal again and studied what she saw. “I can see what they are doing. This is a very verbose way of doing things and I’m too afraid to change it too much. But see this?” She pointed at a part of the code. Nurob had never worked on the Software side of the engine. He was primarily a mechanic, and even though he could read enough to understand what’s going on, he admired her ability to actually make sense of it. “This seems to be… depending on some timing? The timing of that giant magnetic platter in section C of the engine? I know the platter slows down over time, so I assume this part also executes slower over time?” She nodded. “Indeed. This is a common issue on older engines like these. But it is a relatively easy fix with some of the new methods we developed in recent years. See these two things here? They need to happen before we can continue, but they are not dependent on each other. So I can just…” She started typing frantically. Nurob knew the look in her eyes too well. Absolute immersion, focus on the thing one is working on. Being “in the zone”, to use another term he learned from the humans. There are many Guardians in the engine, but most only did what they were told to do, very few had the ability to completely dive into the inner workings with passion and genuine interest. Eventually, she looked at him again. “So this is now executing in parallel, which should buy us another three or four billion earth years to come up with a proper solution. Should we deploy this change to the engine?”
This was the hard decisions a person in his position had to make. It is what everyone expected from him. His job seemed so easy to the other Guardians who were doing the real work, but his decision right now could destroy the project or make it succeed. Success was one of these things no one could measure because it just meant the absence of catastrophic failure. This is what he had to avoid now. But after watching Nahira work, he was confident that this was the right decision. “We’re going to emit some Solar Flares while we’re deploying. That should keep the brief outage hidden. Push it!” he declared.
There was a large observation window in the engine room, and Nurob stood there watching the planet Earth through it. Whenever he looked through one of the windows that were part of the Engine complex, he believed that someone was watching him as well. These primitive mammals with their limited intelligence and capacity, yet with more creativity than any other species he knew. Their enthusiasm for the Universe was what made him love them so much. The Flares would surely give them some headaches and he could already imagine them running around, looking at data and worried about the things they shot into space. And as he stood there and watched, Nurob wondered if someone else was watching him right now, in an Engine that really is powering him and the Guardians?
He knew that he would have another billion earth years to ponder that thought.
Beer. One of the great things about the US is that homebrewing is legal. It kinda needs to be because the prohibition killed off so many American breweries, we’re still living in the aftermath where the beer with the biggest market share is not American at all (It’s owned by a Belgian-Brazilian* company) and also tastes like urine.
But once you start looking past Bud, you realize just how awesome American Microbreweries are. If you’re in SoCal, I highly recommend the Orange County Beer Festival in Silverado to sample the goods that people brew here. If you’re visiting, I recommend anything from Kona Brewing Co, most Sam Adams or a Shiner Bock.
Homebrewing is one of those things that are easy to pickup and start and then offer a lot of room to master it. I started out with a simple starter kit that contained almost everything I needed, except for a 5 gallon kettle and some bottles (5 gallon results in ~45 12oz bottles).
After checking your local laws, you start out by boiling the beer for an hour, putting in any special ingredients that come with the recipe kit and stirring a lot. If you have a lawn and an outdoor burner, awesome! A gas stove works well enough as well, just expect it to take longer, but the risk of nasty boilovers is lower.
After that, you need to chill it as quickly as possible. An ice bath works perfectly well for the start, although once you get serious I’d recommend a Wort Chiller.
You then move the chilled wort into a fermenting bucket, which can be a glass carboy or a plastic bucket. Make sure that stuff is sanitized (the starter kit comes with sanitizing solution).
This then goes into a cold, dark, quiet place for 2-4 weeks, depending on the recipe. Now you can take the time cleaning up the kettle and all equipment while you’re waiting…
After the fermentation is done, you fill the beer into a bottling bucket, together with some sugar (the recipe contains all the details). From there, you then fill the (sanitized) beer bottles one by one. This may be a bit messy as stuff will drip when you pull out the hose. Having another person helps, but isn’t required.
The kit comes with a bottle capper and caps, which is a quite fun part of the bottling process. If you want some awesome custom printed caps, Bottlemark sells them for 12 cents a cap. Might be a good investment for the future.
After you’ve bottled all ~45 bottles, it takes another week or two and you’re good to go! (These beers are bottle conditioned and there will be a small layer of (harmless) sediment at the bottom. It’s recommended to drink from a glass.
This process is a bit simplified, but not by much, I mainly left out sanitizing and siphoning stuff. Brewing beer from a recipe kit is a really simple process, but one that opens the door into a whole world of experimentation and good flavors. You can also grab a few 1 gallon kits to have several experimental brews in parallel. There is also a StackExchange site dedicated to Home Brewing.
The 21st Amendment to the US constitution was one of the most important decisions in the past hundred years, a true victory of democracy. Let’s not desecrate that by drinking Bud.
*Just to be clear, this is no stab against the countries Brazil or Belgium or their beer culture. Especially Belgium has an amazing culture of great beers like Stella Artois. I don’t know much about Brazilian beers, but people told me that Eisenbahn Weizenbier is awesome.
**Drink in moderation. Alcohol can be awesome (I’ll refer to Korpiklaani who said it better than I ever could), but it also has destroyed lives of people who grew too dependent on it. And seriously, if you drink and drive, you deserve to get into an accident that leaves you paralyzed for the rest of your life.
***I think it’s stupid that the legal drinking age in the USA is 21. I come from a country that legalizes Beer at 16 and even though there is still a lot of stupid drinking happening, Germans are known for having mastered social drinking. Ask anyone who’s ever been to a Hofbräuhaus.
February 2nd, 2013 in
Entertainment,
U.S.A. | tags:
beer |
1 Comment
Adventure Games went a long way from the Text Adventures of the 80’s to be something like an anachronism in the 2010’s landscape that’s dominated by twitch-based action games. Here are some GUIs that Adventure Games used over the years. This is by no means exhaustive, but should cover the most common ones.
Early Sierra Games were essentially still text adventures – you could move your character, but not interact with anything without typing.
(King’s Quest 1)
A lot of early games used text-based GUIs that still resemble Text Adventures, except that the parser is now hidden, thus eliminating the “Guess the Verb” problem.
(Zak McKracken / Commodore 64)
(The Secret of Monkey Island / Amiga)
Lucas Arts would eventually reduce the number verbs and replace the inventory with icons.
(The Secret of Monkey Island / PC DOS, VGA Version)
(Indiana Jones and the Fate of Atlantis / PC DOS)
Some games had icons for the actions.
(The Secret of Monkey Island Special Edition / iPad)
(Das Erbe / Amiga)
(BiFi Roll Action in Hollywood / Amiga)
There are Hybrid Icon/Text approaches.
(Das Telekommando kehrt zurück / Amiga)
Assigning Body parts is a way to not have actions per-se (since e.g. a Hand can mean Push, Pull, Punch, Use, Climb, Shoot…),
(Full Throttle)
(Gemini Rue)
(Curse of Monkey Island)
Some games use a simple Interact/Look breakdown:
(Secret Files: Puritas Cordis, but the same cursor was used in Secret Files: Tunguska and Lost Horizon)
(Deponia)
And some games didn’t have verbs in the traditional sense at all, but relied on a single context-sensitive action, plus inventory action and dialogue. However, sometimes right-clicking implies “Look” while left clicking is “Interact”, so it’s not really that different.
(Broken Sword: The Shadow of the Templars)
(Legend of Kyrandia)
(Space Quest V)
A special case are multiple verbs that are context-sensitive (so they may completely differ from hotspot to hotspot)
(A New Beginning – Final Cut)
So yesterday The Verge reported that the 64 GB Surface Pro would only have 23 GB free space left, and all of a sudden the internet pretended to be surprised. Of the many reactions, I think that Marco Arment had the only interesting one, but his “Truth in Advertising” suggestion will likely clash with the ignorance of customers around the world and especially in the USA, so don’t expect anything to happen.
Any why should it? The Surface Pro is a PC, not a Tablet. It has a real, 64-Bit x86 CPU instead of an ARM Chip and it runs a real operating system instead of one of these slimmed down Tablet OSes (Granted, Windows 8 is a vastly inferior version of Windows 7 with a halfway-decent Tablet OS clumsily bolted on, but that’s a different story – it’ll be interesting to see the people always screaming “But we want real Windows apps on our devices!” realizing that almost all real Windows apps until now are meant for Keyboard and Mouse since they have small touch targets, right-click menus and only work well in the default 96dpi).
Have you ever tried to install a real Windows on a 64 GB C: Drive? There is a reason small SSDs aren’t really that popular as boot drives, especially when you consider that the Page- and Hibernation files also take up space (although the small amount of RAM – 4 GB – in the Surface Pro might help a bit).
Once you start installing a few applications, that space will go away just as it would on a normal PC. That’s when you connect an auxiliary storage device (a D: Drive in form of an SD Card) and read up tweaking guides to move stuff around, just like people did when they tried cramming Vista and Win7 on a 64 GB C: Drive.
It will be seen if the Surface Pro is crippled by UEFI Secure Boot like the Surface, thus preventing you from upgrading to Windows 7 or Vista, but if it does, don’t complain since it was your choice to buy such a PC.
I’m not saying that the Surface Pro is a bad product, but you should be aware that you’re not buying a Tablet running an optimized Tablet OS which runs optimized Tablet Apps to give you an optimized Tablet experience. You are buying a Touchscreen Laptop with all Pros and Cons. Which is why the Type Cover is such a good, important and mandatory idea.
January 30th, 2013 in
Uncategorized | tags:
surface |
No Comments
