Just a quick well-deserved post plugging newegg for taking care of me so well. I ordered some open-box hdd’s, clearly labeled as having a refund-only, no replacement policy. One of the disks I received was refurbished. I called to say no, that’s not what I ordered, suspecting a refund would be the only remedy offered. A refund would have bummed me out a little since I ordered 3 disks and ideally that’s how many I wanted at such an uncommonly good price. But, when I told that to the representative (who did initially just offer me a refund), he bypassed several standard policies to ship a replacement disk – before I had returned the refurb so that I would get one before their open-box stock ran out – shipped it overnight, and sent me a prepaid return label as well. The correct product arrived tonight, and all three disks are now original and show 0 hours previously powered on. Way to go Newegg!
ZFS block pointer rewrite status, 2013
Through one part idle internet window shopping and one part good luck, I came across a supply (now sold out) of the disks I use in my main RAID at home, nearly 50% off the next lowest price I’d ever seen for them–open-box (but not refurb) disks from a reputable online retailer with only a few months already consumed on Seagate’s 5 year warranty. This seemed to me by far the best deal I’d seen on these disks in 5 years of looking, so I immediately bought some.
So now I am at one of those rarely encountered times where I have sufficient unused hard disk capacity to create a whole second copy of my current stash of bits if I wanted, and that gives me the flexibility to switch to another disk/filesystem/volume management arrangement if I want.
I always assume that at some point the amount of data I’ve accumulated will grow too large to allow for another opportunity to start from scratch like this, and so even though I’ve tried to choose wisely in selecting a storage arrangement in the past, and as a result could feasibly grow my current MD-based RAID out to 22 TB or so, I still want to make sure there isn’t something better out there that I could switch to since I do have the chance to do it now. Plus, the last time I investigated storage options was three years ago (which put me on my current 2TB-per-member MD RAID with ext4), so I took another look.
There are lots of things I like in ZFS, such as the added data integrity safeguards, snapshots to protect me from the fat-fingered rm -rf *, and with support for remote mirroring, it would definitely be worth another look at nearly-realtime backups to my hosted server (something I previously rejected due to the lack of snapshots.) A less unique but still important feature for long-term scalability is logical volumes, as I think a single RAID with > 12 drives would be a stretch even for the relatively light load my home array experiences.
But, as many a home user before me can tell you, ZFS has one huge feature hole for nerd@home scale operations: it is not possible to drop in another disk or two onto an existing RAID set. Enterprise users will drop another rack of raid sets into an existing zpool, which is possible with ZFS and makes sense. But do that after buying a handful of disks, and you’ll have a bunch of little RAID sets with wasted parity disks everywhere. RAID reconfiguration is something that’s been possible with MD RAID since I fired up my first array in 2003, and became possible as on online operation some years afterwards. It’s a feature I’ve used several times that I’m not comfortable giving up.
So, I dug into the state of development of this for ZFS, and was pleased to find some pretty current and comprehend-able information from Matt Ahrens, co-founder of ZFS way back when it was at Sun (awesomely, he seems to still be involved in ZFS through employment elsewhere.) The short summary sounds like achieving this via “block pointer rewrite” will almost surely never happen, because any code that correctly updates all necessary data structures would, in Matt’s opinion, introduce code that is too confusing/ugly/unmaintainable to encourage further ZFS development of other kinds. The YouTube video from October 2013 that I found this on is a great watch, if you’re interested in more detail. He also says, however, that there may be less ambitious approaches than block location rewrite to achieve the similar task of removing a device from a pool, so perhaps the more targeted task of adding a disk to a RAID-Z could also be tackled another way. Something I might try to learn more about?
First though, I need to research ZFS basics a bit more to find out why I shouldn’t just build a zpool of one or two vdevs (configured from ZFS’s point of view as plain disks) that actually happen to be MD RAID devices. Would it work to just let MD RAID do what it does best, and let ZFS do what it does best on top of it?
(Update 1/2014: the above md/zfs scheme would be ill-advised because ZFS implements a nontraditional “RAID” that includes couplings to the filesystem level, providing advantages such as self-healing data in the event of a parity/data disagreement, since the “RAID” subsystem and filesystem can compare notes to see if the data aligns with ZFS’s filesystem-level hashing (meaning the parity must be bad) or does not align with ZFS’s filesystem-level hash, meaning the data should be reconstructed from parity.)
JavaScript’s Object.keys performance with inheritance chains
Backstory: (You can cut to the chase) I’m working on a JavaScript library in which I want to identify any changes a method has made to properties of the object it is applied to, preferably in a more intelligent way than cloning all the properties and comparing them all before and after invocation. This can be done by applying the method in question to an empty object x inheriting from the true/intended object, and looking for any properties in x after the method is applied.
But, how does one efficiently look for the properties in x? In JS implementations supporting ECMAScript 5, Object.keys looks like a promising candidate. The function is decidedly O(n), but I wanted to be sure that with ES5, “n” is the properties of just my object, and not properties of all objects in the inheritance chain, like the ES3 polyfill.
(The chase:) My test on jsperf.com shows that yes, in browsers with ES5, Object.keys doesn’t waste time iterating properties of the inheritance chain and discarding them.
See also a simple html page with some primitive timings, which basically says the same thing:
<html>
<head>
<script>
function BaseClass() {}
for (var i = 0; i < 10e4; i++) {
BaseClass.prototype["p" + i] = i;
}
function SubClass() {
this.thing = 'x';
}
SubClass.prototype = BaseClass.prototype;
var objInTest = new SubClass();
function polyfillKeys(obj) {
var output = [];
for (var key in obj) {
if (obj.hasOwnProperty(key)) {
output.push(key);
}
}
return output;
}
// could do Object.keys = polyfillKeys
var start = new Date();
var keys = Object.keys(objInTest);
console.log("Native Object.keys: " + (new Date() - start));
start = new Date();
var keys = polyfillKeys(objInTest);
console.log("Polyfill way: " + (new Date() - start));
</script>
</head>
<body></body>
</html>
Onkyo TX-NR414
I’ve owned the Onkyo TX-NR414 AV receiver for a year now, and thought I’d branch out the subject matter of this blog a bit with a product review of it. My comments come more from a feature perspective than an audio clarity standpoint, because I’ve pretty thoroughly exercised its feature set, but am not a certified audiophile. What I will say in terms of audio quality is that it delivers if you are someone who can appreciate the sound of an above-average system (average being one of those $250 department store surround-in-a-box kits.) I’ve got it connected to two Bose 301’s for front speakers and two wall-mounted Bose 161s for surround; this configuration produces clear and undistorted audio up through volumes that doubtless disrupt the neighbors.
The features, on the other hand…
Pros:
- It attempts to implement a lot of features. Many look good on paper, but I’m not listing the flaky ones in this category.
- Supports power on/off commands over the network. This feature works reliably. (Through the mobile app; I did a quick trial of standard WOL unsuccessfully.)
- Wealth of surround calibration and configuration options to support most any speaker configuration (including no front center, as in my case.) Easy to fine tune levels and delays.
- Has comprehensive and searchable shoutcast/icecast-basd internet radio listing; integrates with other online music services that I haven’t tested it with.
- Internal microcontroller never freezes, crashes, or otherwise totally spazzes out so bad that it needs a power cycle.
- It supposedly has better DACs than you’d find on PC motherboards etc; I like to take them at their word when listening to lossless audio with my eyes shut in a darkened room, though in reality I doubt I’d actually notice the difference.
Cons:
- HDMI pass-through ports lose some visual fidelity. This isn’t really noticeable with anything less detailed than text from a PC or 1080p device generating text around 12pt. I’m not sure if its running some image processing algorithm or what, but it almost looks like it’s anti-aliasing the text – the text’s color influences the color of the adjacent pixels a bit. This doesn’t happen if I connect the devices directly to my TV.
- It attempts to support HDMI-CEC to control the power of the TV and change inputs, volume etc based on actions taken by connected HDMI devices. This could be done much more intelligently. It works well enough for TV/video watching that I leave it enabled. Turn on the TV, and odds are the TX-NR414 will automatically power up, switch to the TV input, and command the TV to mute its internal speakers. I say “odds are” because it doesn’t always power the receiver up…no rationale I’ve identified as to when it does/doesn’t work. However, it proves to be downright frustrating when I want to listen to music. A typical scenario might go something like this:
- I’d like to listen to the radio. I turn on the receiver, which was last used with the TV and is on TV input. I immediately switch inputs to FM. But, the receiver already commanded the TV to power up. As with all TVs nowadays, this isn’t an instantaneous process; my TV comes up maybe 6 seconds later. Presently, the TV commands the receiver back from FM onto TV input. I now have a TV powered up and pulling ~500 watts and am no longer listening to the radio. I manually switch back to FM. Everything’s now fine. Then I turn the TV off. The TV commands the receiver to power off. Arg.
- In a similar scenario, I’d like to listen to music via a CIFS share on my home network, selecting songs via the Android app. (Yes, with much pulling of hair, you can stream music straight off windows file shares and/or samba. Most every audio codec is supported – it’s about the only device I know of that doesn’t run Windows but does play WMA-Lossless. But everything else is too rudimentary/buggy for this to make the pros category.) I tell my phone to switch the receiver to the “net” input, which immediately powers on the TV – totally unnecessary since I’m browsing the music on my phone.
- Audio files from CIFS shares aren’t searchable or even sortable, and are listed on the Android app by the song’s title in the file’s metadata. My samba server feeds it the files in approximately alphabetical order by artist, so I have thousands of apparently unordered song titles listed on my phone to scroll through. I know my music collection well enough to generally figure out which artists go with which songs and determine where in the list I am, but it’s way trickier to find a song than it should be.
- Since the last firmware update, there seems to be a bug when selecting either a CIFS or DLNA server to connect to via the android app. You tap the server to connect to and it never connects to it – you must hit enter either on the infrared remote or physically on the receiver to proceed.
- Infrequent firmware updates to improve on any of the above issues. Firmware updates that do occur sometimes drop all user data (radio presets, input names, surround config, network bookmarks, …) without warning.
Uber-geeks might be interested to know that nmap says it’s running AXIS Linux 2.6, and that it has a basic web interface served by lighttpd, allowing configuration of network settings and input of favorite internet radio stations.
So, it handles all the absolute necessities of a modern surround receiver. It’s good enough that I don’t foresee springing for any “upgrades” until it fails. But, all the less critical frills like the Android app, home media streaming, HDMI passthrough and HDMI-CEC aren’t totally ironed out.
Addendum, Nov. 2014: The receiver stopped working last month, five months out of warranty. I was heading out the door to the repair shop when I thought I better check Google quick for anything first. Onkyo had launched a special service program for the particular failure I was experiencing just two days before. They required repair to be completed at a central shop of theirs, but sent me a box and packing materials, paid for shipping both ways, and repaired the unit at no cost. Looks like the new component they put in also cleaned up the HDMI quality issues. Overall, this experience had the desired effect on my customer loyalty for the brand.
Sendmail’s unwanted insistence on local delivery
Here’s another quick post to record one of those 2-line solutions to a problem that took considerable searching to find. This one affects outgoing mail passing through sendmail when the recipient’s email address matches the machine’s hostname, but the machine is not the mail server for the domain. For example, my dedicated server is configured as mbaynton.com, and sends logs and such to my @mbaynton.com email. Sendmail on this machine works fine for emails to other domains, but the trouble is, as every other email server in the entire world besides mine knows, mail destined for addresses @mbaynton.com should be sent to the smtp sever at aspmx.l.google.com. Instead of looking up the mx record for mbaynton.com to see if it actually is the mbaynton.com mail server, it just assumes it is, looks for the recipient’s name in its local user database, and either delivers the message locally or doesn’t find a local mailbox and gives up.
The fix: add the following two lines to the end of /etc/mail/sendmail.mc:
define(`MAIL_HUB', `mbaynton.com.')dnl define(`LOCAL_RELAY', `mbaynton.com.')dnl
Then rebuild the sendmail configuration, which on Ubuntu can be accomplished by running
sendmailconfig
Since sendmail is going to be with us for the foreseeable future, I’m sure I’ll need to refer back to this tip someday. Thanks to http://serverfault.com/questions/65365/disable-local-delivery-in-sendmail/128450#128450 for the solution.
Nifty openssl troubleshooting trick
Learned a neat trick today for performing diagnostics on failing ssl connections when working with programs making use of the openssl libs that are notorious for providing crappy ssl error information. You can use the openssl executable to setup a simple test client/server communication, and see any errors that occur in the process:
1. The server:
openssl s_server -cert /path/to/server-cert.crt -key /path/to/server-cert.key -www
2. The client:
openssl s_client -CAfile /path/to/ca-cert.pem -connect 127.0.0.1:4433
These commands print out various information when establishing or attempting to establish a connection, in my case reminding me that my cert had expired (which mysql clearly communicated to me by saying “ASN: bad other sigature confirmation”).
PHP’s mysqli::reap_async_query blocks
Just a quick note about mysqli::reap_async_query since the official documentation is surprisingly lacking, and I’ve never had any luck getting user-contributed comments posted to the php pmanual.
The manual has this to say about mysqli::query(“…”, MYSQLI_ASYNC): “With MYSQLI_ASYNC (available with mysqlnd), it is possible to perform query asynchronously. mysqli_poll() is then used to get results from such queries.”
Does this mean that the only safe way to call reap_async_query is to first poll for connections with complete queries, and only call reap_async_query on connections that you know have results ready? No. Here’s a quick sample script to show what happens if you skip mysqli_poll() –
<?php
$dbhost = "localhost";
$dbuser = "someuser";
$dbpass = "somepass";
$dbschema = "db_name";
$mysqli = new mysqli($dbhost, $dbuser, $dbpass, $dbschema);
$mysqli->query("SELECT SLEEP(5) AS sleep, 'query returned' AS result", MYSQLI_ASYNC);
echo 'Output while query is running...<br>';
$result = $mysqli->reap_async_query();
$resultArray = $result->fetch_assoc();
echo 'Got back "' . $resultArray["result"] . '" from query.';
outputs (after 5 seconds):
Output while query is running...<br>Got back "query returned" from query.
So, it appears that reap_async_query will block waiting for the query running on the mysqli instance to complete, if that query is not ready yet. So, in many cases, there is no need to use mysqli_poll() at all.
Making Ubuntu resolve names like Windows
There are a few oddities in Ubuntu’s name resolution code. They mean well, but in my experience don’t always play well with Windows systems, corporate VPNs, etc. Here’s a few changes I make so that all names resolve identically on my Ubuntu systems as they would on my Windows systems. This post is a note to self next time I setup an Ubuntu system.
1. Ununtu 12.04 added dnsmasq into the standard desktop distro, and uses it as the nameserver by default. This is probably a good thing for the majority of users and use cases, but if you’ve either already setup dnsmasq on a system when you upgrade to 12.04, or already run dnsmasq on a central server for the network your 12.04+ system is on, then you might want to get rid of the new dnsmasq that’s embedded in the Ubuntu distro. Fortunately, it’s easy to do: just comment out
dns=dnsmasq
#dns=dnsmasq
in /etc/NetworkManager/NetworkManager.conf.
2. Ubuntu tries to do a magic name resolution of systems within IP-broadcast shouting distance (using Avahi) when the name provided ends in “.local”. As a result, if you’ve actually got names on your network ending in .local that an upstream DNS server is configured to answer, and those hosts aren’t running Avahi or are on the other side of a router, then the name resolution will mysteriously fail. I had to bring in Wireshark to figure out what the hell was going on for this one. The fix, mercifully, is about as easy as the config change in #1 – just tell Avahi to intercept some magic TLD other than “.local” by setting
[server] domain-name=.home
in /etc/avahi/avahi-daemon.conf.
Securing SignalR to your site’s users
As you know if you read my last post on hub authorization, when a new hub connection is created, the transport connection stays open whether or not the hub connection(s) are authorized. In my case, I am using SignalR in a carefully controlled, secure authenticated environment, and as a common-sense security measure, I wanted to just close down any connections to SignalR from random clients that had no business connecting to my servers. Unfortunately, as of 1.0 RC1 there’s no built-in way to do this, but with some quick pointers from David Fowler I was able to put together something with Owin that causes my code to be run on every request first, before handing it off to SignalR. The result seemed like it might be useful enough to other people to be worthy of a blog post.
Disclaimers: This code was written for SignalR 1.0 RC1 and the version of owin that was around at that time. A visitor has posted some updates in the comments on this post’s page for SignalR 1.1.3. This applies to SignalR on top of IIS; I don’t know how it would be different if you’re hosting in some other way. This applies to requests that are routed to SignalR hubs (hub transport connections, method invocations, hubs auto-javascript.) You may need to do a similar thing for Persistent Connections too if you use them. Also, most of the calls and callbacks to and from Owin are just what “seemed to work,” but that doesn’t mean this is the best or most correct possible way to do it. We’ll just have to wait for Owin to be documented for that.
Step 1: Ditch RouteTable.Routes.MapHubs();
You should recognize this call from your application startup code (if you added SignalR via nuget, it’s in App_Start/RegisterHubs.cs). Internally, MapHubs tells Owin to route requests with certain urls right into SignalR code. This isn’t how we want to route. What we want to do is route them to our code first, where we’ll decide if we want to let the request through to SignalR or close it. So, we have to do do some routing calls ourselves:
// in addition to the usings you already have,
using Microsoft.AspNet.SignalR.SystemWeb.Infrastructure;
using Microsoft.Owin.Host.SystemWeb;
using Owin; // actually defined in a SignalR assembly and probably going to change soon...
public static void Start()
{
// RouteTable.Routes.MapHubs(); not doing this anymore
MapHubsWithSecurityInspector(RouteTable.Routes, "~/signalr", GlobalHost.DependencyResolver);
}
private static RouteBase MapHubsWithSecurityInspector(RouteCollection routes, string url, IDependencyResolver resolver)
{
var existing = routes["signalr.hubs"];
if (existing != null)
{
routes.Remove(existing);
}
var routeUrl = url.TrimStart('~').TrimStart('/');
var locator = new Lazy(() => new BuildManagerAssemblyLocator());
resolver.Register(typeof(IAssemblyLocator), () => locator.Value);
return routes.MapOwinRoute("signalr.hubs", routeUrl, map => OwinSetup(map, resolver));
}
private static void OwinSetup(IAppBuilder builder, IDependencyResolver resolver)
{
builder.Use(typeof(MySecurityInspectionHandler));
builder.MapHubs(resolver);
}
Step 2: Write MySecurityInspectionHandler
After you’ve replaced MapHubs with the above, Owin will need you to define a MySecurityInspectionHandler class, because it will try to call it on every incoming request to routeUrl. The signature of MySecurityInspectionHandler is a bit magical, as the constructor and callback Owin actually looks for isn’t expressed anywhere in docs or as an interface or anything. Here’s what seems to work:
using System.Threading.Tasks;
// u might want to put the rest in your app's namespace...
using AppFunc = Func<IDictionary<string, object>, Task>;
public class MySecurityInspectionHandler
{
private AppFunc _app;
public MySecurityInspectionHandler(AppFunc app)
{
_app = app;
}
public Task Invoke(IDictionary<string, object> environment)
{
// Finally, here's where we can examine the incoming request and allow or reject it
if(RequestHasRightCookiesOrSomething(environment))
{
// continue processing the request
return _app.Invoke(environment);
} else {
return StopRequestProcessing(environment);
}
}
private Task StopRequestProcessing(IDictionary<string, object> environment)
{
environment.Add(ResponseStatusCode, 403);
object responseStreamObj;
environment.TryGetValue(ResponseBody, out responseStreamObj);
Stream responseStream = (Stream)responseStreamObj;
var streamWriter = new StreamWriter(responseStream);
streamWriter.Write("403 Forbidden.");
streamWriter.Close();
var tcs = new TaskCompletionSource<bool>();
tcs.SetResult(false);
return tcs.Task;
}
}
Then all you have to do is implement your actual logic to detect if this is a request from an authorized source or not, in a RequestHasRightCookiesOrSomething method. Just about everything you could ever want to know about the incoming request can be found in that environment IDictionary, unfortunately as objects with string keys that Intellisense couldn’t help you with. When you get to this stage just shove a breakpoint in and take a look at the Keys / Values inside environment to find what you need.
Step 3: Test
Visit some urls to your SignalR hubs manually in a browser, both with and without the expected authentication features sent along. When you do not send the authentication information, you should get a 403 forbidden back from URLs like
- http://localhost/myproj/signalr
- http://localhost/myproj/signalr/hubs
- http://localhost/myproj/signalr/anything_else
When you do send authentication, /signalr visited manually currently produces an unknown transport exception, and /signalr/hubs produces the auto-javascript.
Hope this helps. Happy routing!
SignalR Hub Authorization
I set about making use of the new hub authorization feature in SignalR 1.0 today. It was a bit difficult to obtain answers about what it actually does and how it works, so I studied the revision that introduced this feature, wrote some test code of my own, and thought I would post my findings. This applies to the current version of SignalR as of this post, which is 1.0 RC1.
Some important Hub Authorization bullet-points:
- Authorization to connect to a hub shouldn’t be confused with authorizing the underlying persistent connection. This means if a client (I happen to use the .net one) connects to two hubs with
// Create a proxy to the chat service var hub1 = hubConnection.CreateHubProxy("HubTypeOne"); var hub2 = hubConnection.CreateHubProxy("HubTypeTwo"); // Start the connection hubConnection.Start().Wait();and both HubTypeOne and HubTypeTwo do not authorize the hub connection, the underlying transport connection still stays open. This does make sense if you think about it, for WebSocket transport at least, since being authorized to connect to a hub is not a prerequisite to being authorized to invoke methods on the hub. But, an important consequence of this is that hub authorizers offer no protection against random people/bots that want to open up zillions of TCP sessions to your server, or attempt other shenanigans. I’ll have another post discussing how to authenticate all requests to SignalR before they even hit SignalR code soon – I’ve done it today, but have to write up the post…
- Code samples using the Authorize attribute are in the SignalR source at samples/SignalR.Hosting.AspNet.Samples/Hubs/Auth/. See also the hub authorization feature introduction in David Fowler’s blog.
- If you don’t implement any SignalR interfaces and just use the [Authorize] attribute, then the actual authorization logic that runs (code from Microsoft.AspNet.SignalR.Hubs.AuthorizeAttribute) is
private bool UserAuthorized(IPrincipal user) { if (!user.Identity.IsAuthenticated) { return false; } if (_usersSplit.Length > 0 && !_usersSplit.Contains(user.Identity.Name, StringComparer.OrdinalIgnoreCase)) { return false; } if (_rolesSplit.Length > 0 && !_rolesSplit.Any(user.IsInRole)) { return false; } return true; }where
- user is the System.Security.Principal.IPrincipal associated with the connection or hub invocation. If you know what what this is great, otherwise do what I did and write your own authorizers that tie into your app’s user management (see below.)
- _usersSplit and _rolesSplit correspond to the attribute’s Users and Roles properties, allowing you to taylor the authorization to specific usernames or roles:
[Authorize(Roles="Admin, Poweruser")] public class MySuperHub : Hub
- There’s a shortcut extension method to apply authorization to all hubs (any authenticated user, can’t specify usernames/roles):
GlobalHost.HubPipeline.RequireAuthentication();
This should be called in your application startup code before creating your hub routing (MapHubs()).
Implementing your own authorizers
If you don’t like the UserAuthorized method that is the heart of the [Authorize] attribute, you can write your own authorizers. To do this, create a class that implements at least one of Microsoft.AspNet.SignalR.Hubs.IAuthorizeHubConnection or Microsoft.AspNet.SignalR.Hubs.IAuthorizeHubMethodInvocation. The parameters to these interface’s methods are very sensible and provide all sorts of information you might want in making an authorization decision – hub, method, user, cookies, and many others. If you want to apply your authorizer to hubs or hub methods by decorating them in code, you’ll of course need to subclass Attribute too. Here’s a class declaration that does all three of these things to get you started:
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple = true)] public class CustomAuthorizeAttribute : Attribute, IAuthorizeHubConnection, IAuthorizeHubMethodInvocation
Applying your own authorizers to all hubs and method invocations
var globalAuthorizer = new CustomAuthorizeAttribute(); GlobalHost.HubPipeline.AddModule(new AuthorizeModule(globalAuthorizer, globalAuthorizer));
Again, this should be called in your application startup code before creating your hub routing (MapHubs()).