Programming :: Even One Word

Posts Tagged as ‘programming’

The New Culture of Spam

Feb 03 2011

I realize I’m not the first one to talk about this, (see Cory Doctorow’s thoughts on the subject), but the nature of the internet and spam have changed over the years.

It used to be that you protected your e-mail address as though it were the most secretive information in the universe, next to the last four digits of your social and your weight (and/or age, depending on your gender and personality). But, just like the last four digits of your social, your weight, your age, and even probably about how much money you make, your info is all over the freaking place. But this isn’t just the case on e-mail, it is the case with blogs, forums, and any form of communication which would ever require a turing test. As of right now, I have about 200 comments on my blog, and only 4 of them are legitimate. So how did people fight this?

Obfuscate your e-mail address like name (at) domain (dot) com.
Turn off comments on your blog.
Employ a site admin, moderator, or other full/part-time culler of the wheat from the chaff (or the spam from the real meat, whichever analogy fits your dietary restrictions best).
Use the aforementioned turing test anywhere that allows human input.
Automation.

The problem we have with every one of these is that they are all treatments for symptoms of an underlying disease. As far as which of these symptom treatments works best?
The shortest and last one, of course! The 200 comments on my blog, around 190ish of which are marked spam, were filtered by a plug-in most wordpressers know as Akismet. My public e-mail (nathan@nathanstpierre.com, btw) is in no way obfuscated or filtered, because it goes through g-mail, which has (as far as I’ve seen) the most intricate spam filtering system available to a public mail system. As Mr. Doctorow so intellectually pointed out, these systems exist for exactly this reason, so why not utilize them?

That being said : I still think the underlying disease goes untreated. Unfortunately, this disease is the same one that ruined the old-school forums, social media (MySpace whores anyone?), and even legitimate community tools (buy something from someone who’s not a Nigerian in Craig’s List. I dare you.). It is the chief weakness and strength of the Internet: freedom. The freedom of an open and endless system is that you end up ultimately having to be at the mercy of the demographic that utilizes it.

Great examples of the successes of this philosophy include Open Source (none of us is as smart as all of us), Wikipedia (the nice people who care will ultimately win out over the jerks, because they are not doing the easy thing, they’re doing the thing they care about). The failures of this freedom are pretty much the inverse: Reporting of false information on national TV thanks to the Bogus Blogosphere, entire systems being overrun by spammers (Google groups anyone?) and so forth. So is there a cure?

In the spirit of presenting a solution rather than a problem, I suggest a change. Not a change in software or business models, a change in philosophy. We once thought the internet was too massive and too free to infringe upon, but YouTube shattered that preconception to me when they freaking scanned a video for copyrighted material. Could software ultimately determine what’s spam and what’s legit and be a part of every ISP’s basic network protocol, insta-deleting anything that clutters their domains with horrendous spam? Potentially. But should it?

Honestly that kind of big brother dystopia –which would likely lead to my favorite cyberpunk plots being possible– makes me think that’s the more mechanical answer, which completely ignores the spirit of the issue, which is a cultural question. The culture of the internet has become beneficial to spam.

But this is a blog about web development! you say? How does being a hippie and talking about working for a new society help anything?

Well, algorithms are great, but as Google is finding out, they’re not the answer to everything (or what we call a silver bullet). For more information on an example where someone gamed Google’s algorithm in a seriously negative manner, check out the story of DecorMyEyes. To summarize: a shady businessman discovered that Google ranks things based on how often people mention your site, along with certain search terms. So he discovered that people blasting him on a thousand ripoff sites about his failure to manage their (insert glasses brand name here) order the correct way, would cause his site to show up first for someone looking for (insert brand name here) and/or “glasses.” Google’s response? Essentially, change the algorithm (to see their actual response read here).

… in the last few days we developed an algorithmic solution which detects the merchant from the Times article along with hundreds of other merchants that, in our opinion, provide an extremely poor user experience. The algorithm we incorporated into our search rankings represents an initial solution to this issue, and Google users are now getting a better experience as a result.

Is this a good solution? Honestly, it’s probably the best solution given the situation. They explain this in the article, but they point out that just blocking this person or using sentiment analysis (filter of good vs. bad reviews) could cause the inverse problem to happen: game the system and post a million bad reviews of Best Buy and suddenly they never show up in Google searches for Best Buy.

But what’s happened pretty recently with them and Microsoft’s Bing makes me think the algorithm isn’t the solution, it’s the problem.
If you haven’t heard the latest news, check out this article from Seattle’s own KIRO TV. Essentially, Google set up a “honeypot” by putting out some completely random result sets for random character searches, and Bing turned up the same results. Now unless they figured out how to steal one of the most carefully guarded algorithms in the tech industry, I highly doubt this would happen as a freak accident. It’s pretty clear Microsoft is doing something sketchy. Whether they are or not, let’s say someone at some point did. This would prove my point: the world ultimately doesn’t care whose algorithm it is. If you can steal it, where’s the incentive not to?

So we come back to the issue: the culture of spam. As YouTube discovered (I’m sure through Google’s technology), there are ways to automatically figure these things out. As I said before, it was probably the best of the options we have at the moment, but we honestly need to find a better way to approach this. For this, I go back to what I mentioned earlier: f*cking Wikipedia and open-source: how do they work?

They work by having the appropriate balance of resources, both personnel and technology. Enough coders are willing to clone your git repository of a new build and try to break it. This is hard and challenging. This scares off spammers, who will try to take the easiest and possibly fastest route through the maze to the cheese. People who are earnestly devoted to a cause will always inevitably find a way over people who are lazily employing practices that work by gaming systems. Why?

Ask the Russians, who spent the entire cold war stealing and duplicating western technology to master and decrypt it just in time to be three generations behind their innovating enemies. The Black Sunday Kill is a better example of this in action in the technology world.

So ultimately, what is the exact mechanism by which we can make search engines, blogs, e-mail and so forth unspammable? I don’t know. Not that I’m incapable of figuring that out, I think someone at less than my skill level can easily figure this out given enough incentive (usually motivation like anger and resources like free time). But the ultimate solution will be counter-acting the current disease: spammers are making money.

Every one of the ads you get that advertise “A bigger Pen15″ makes money, because someone who got that e-mail sent money to someone. Every time a Nigerian princess is ransomed in your e-mail, some gallant fellow cashed out his 401(k) to save her. That one man’s $5,000 is worth orders of magnitude more than the cost of sending out those spams mails, which could vary from a few hundred dollars for millions of e-mails to a dollar for thousands (depending on the location of the servers and the botnet being used). There are lots of resources that discuss this, but my favorite right now is the HowStuffWorks explanation.

So the solution seems simple enough, just keep them from making money! But how do we approach this?

Well, we’ve tried education on massive scale: from teaching your grandparents not to click on spam to educating your children through computer literacy about scamming. We’ve tried blocking those parts of the internet from people to protect them from themselves. We’ve tried spam-blockers, captchas, and every automation system possible. But these ALL address the symptoms. Even trying to legislate against spam has ended up being a pipe dream (and honestly legislation just makes breaking laws more fun for those who’d want to do that in the first place). My proposed solution at this moment?

Make them pay.

Legislation proposes fines, but the problem with legislation is it’s only justifiable if we can prove beyond a shadow of a doubt that suspected perpetrators are in fact perpetrators. I say, we employ the same annoying bastard tactics that we saw in use against the enemies of Julian Assange. Will it be easy? HELL NO.

As we all know, spammers utilize botnets and hordes of zombies, so tracking down all of those spam-emails will most likely lead you to victims instead of perpetrators. On top of this, they often launch attacks such as DDoS or ping-storms to people who attempt to track them down. But these are issues we can address. When these hackers take over a computer, they always leave a back door in order to access it for their various uses. They even come with a kill switch so that they can revert the server/computer/device so others can’t use their system against them. Most of these systems are freely available, and finding the kill switch is just a matter of knowing which attack was launched against your site. They hide their location and just send an anonymous text or e-mail or packet of data to the zombie herd… which makes for a perfectly good honeypot. Intentionally leave a site open to RFI attack, for example, and then monitor any packets that come into the system. When one does, you can find the source. Most likely, it’ll be from behind a series of hops, firewalls, and other zombies, but now you at least know where that came from. Repeat this process enough times, and you find the root. At the very list, you can sniff out the net and either selectively block it, or keep a database of ip addresses logged with what software compromised them so when you find a killswitch, you have somewhere to attack.

Granted these are all ideas off the top of my head, but I’m only one man. And none of us is as smart as all of us, so let’s do this. Let’s get pissed, let’s get serious, and let’s change the culture of spam. Honestly, I think it’s about time the white hats did something other than just turn up their noses at software piracy. And I’m ready to help.

Honesty

Jan 31 2011

So my resolution this New Year was perhaps a bit too broad. I had basically three different parts of my life I wanted to improve: my health, my career, and my creativity. All of these I found to be in some ways expressions of who I am, who I was, and who I want to be. For my health, I decided to make more of a concerted effort to follow a diet, which in this case was weight watchers, because I can simply count the various different things I’m putting in my body to hold myself accountable for my actions. I also wanted to work out more often in order to get into a basically “good” shape, and I didn’t specify to myself how I would do it, other than guarantee I would work out at least three times a week, no matter what days or order they came in. For my career, I decided I would push myself to do things that scared me, including working harder than I’ve ever worked before, and be willing to make drastic decisions regarding my career path. Finally, for my creativity I decided I would read, write, compose, or practice an instrument at least once a day.

So far, I’m amazed at how well I’ve kept up with all of these things. And the reason, I’ve realized, is because I’ve been truthfully, brutally honest with myself. I haven’t been as honest with everyone else, so I’ve decided to start sharing this stuff on the most public and potentially embarrassing place possible: the internet.

In the last couple of months, I’ve:

worked out at least three times a week, in some cases five
lost about fourteen pounds
quit my job (because I realized I wasn’t doing what I wanted to do and accepted a job closer to where I wanted to be)
written three short stories, over thirty poems, three pieces of music
practiced piano, bass, guitar, and tin whistle

In spite of all of this: I feel like I have not yet come close to the spirit of my resolution. Why?

I have lacked honesty. I have lacked truth and the ability to express it in my personal and professional lives. Recently, I read an interview with Francis Ford Coppola, and it’s been running through my mind a great deal in the past couple of days. He said two things that really struck me, so I’ll just quote them both and then go into why they’re so important to this realization.

In the old days, 200 years ago, if you were a composer, the only way you could make money was to travel with the orchestra and be the conductor, because then you’d be paid as a musician. There was no recording. There were no record royalties. So I would say, “Try to disconnect the idea of cinema with the idea of making a living and money.” Because there are ways around it.

What’s the best piece of advice you’ve given to your children, inside and outside of the industry?
Always make your work be personal.
And, you never have to lie. If you lie, you will only trip yourself up. You will always get caught in a lie. It is very important for an artist not to lie, and most important is not to lie to yourself. There are some questions that are inappropriate to ask, and rather than lie, I will not answer them because it’s not a question I accept. So many times we are asked things in our work or in life that you want to lie, and all you have to do is say, “No, that is an improper question.”

I wake up some mornings and wonder where I am, or more importantly how the hell I got here. I graduated one of the hardest music composition programs in the country in four years with honors and no debt, and I was so completely sure throughout that time that I would be destined for musical greatness that I never really paused to think about what that even meant. I saw myself living in a loft in the city with nothing but my bass to keep me company, hunched over scores, or perhaps against the glow of a flickering crt monitor plugged into a desktop on its last legs as I struggled to create my true art.

And here I am, sitting comfortably in a suede chair in the living room of my two-story home, staring at an embarrassingly expensive gaming laptop and writing about my wasted potential. I have become a parody of myself, and I couldn’t figure out why for the longest time.

Now don’t get the wrong impression, I’m not wealthy; I’m barely living paycheck to paycheck against a mountain of student and personal debt, some of which I inherited from my wife, some of which I racked up before I was responsible enough to not live above my means. I have this house because of a government tax credit and a loan from my parents, and I am on this laptop because my company was willing to finance it for a year with no interest for me. As a matter of fact, this comfy sofa is a hand-down from my parents (it didn’t go with their new wood floor). BUT: I write code for 8+ hours a day, most of which thus far powers completely deprecated and inefficient systems in an industry I simply have very little to no interest in (disclaimer : anyone who works for any successful company in any industry knows their app is a kludge, and it’s probably a very profitable kludge). Anyone who knew me from sophomore year of high school to my graduation from UNT would be amazed that I haven’t spontaneously combusted in irony, or that the word “sellout” is not branded to my forehead.

But now I have to step back and realize exactly what has happened and who I’ve become. I’m actually sitting here, for real, and I can actually see and feel all of these things: so this is not the illusion. The dream of working on movie scores or video game music and being a respected musician was the lie. My true art would be laughed out of Hollywood or any “serious” game industry professional’s office. Why? Because it’s not honest.

I think the problem never had to do with me not having the skills or dedication. My ultimate, stinky, sweaty fear under all of those pretty and dressed up excuses was that I would be bound to a lose/lose conundrum. Either I would suffer for an eternity for no ultimate success or reason, or I would be vastly successful and hate myself for what I had allowed myself to become. I would have twisted the thing that has inspired my deepest reserves of personal passion and dedication into some kind of commercialized monstrosity in order to survive to make more, or I’d starve to death (which when you’re married is actually killing two people, more if you have kids). And then it struck me: this sudden clarity came from my personal dedication to this new resolution.

I was honest with myself about my weight. I didn’t feel attractive, healthy, or energetic any more. Being married, you stop worrying about attracting the opposite sex nearly as much, but deep down you’re the same insecure squirming kid you were in the seventh grade, hoping that no one noticed you just pick a wedgie. And how better to better myself than to devote my time and energy to honestly doing the things I’d always wanted to do? I bought a heavy bag (which I’ve wanted since I saw Rocky as a kid), started hitting the gym and the exercise bike and I’ve felt leagues better because I finally told myself the freaking TRUTH: Nathan, you’re a fatass. Do something about it.

I was honest with myself about my career. I got praise at almost every review, and was constantly being told by my co-workers that my input was needed and valued on almost every aspect of development. They told me that I was being considered for a senior position, to be a decision-maker on the system, and I was amazed at how much that failed to inspire me. I finally was honest with myself and asked a very important question: if you work these sixty hour weeks for another year and make it to a senior developer position are you still going to be in the same incredibly restrictive industry, doing business logic that makes people fall asleep when you explain what you do for a living? Being brutally honest, I said yes. So when a friend said his company was looking to fill a designer/front-end developer job, I had to admit it was time to make a change. A terrifying and potentially disastrous (for me) change. And I did.

I was honest with myself about my talent. I told myself for so long that I simply didn’t have the time to work on new designs, write new stories and songs, and practice one of the more than ten instruments I have lying around in the house. I would pine for the opportunity to go and play them or sit down and write, and every time I would stare at a blank screen or just noodle around with songs I’d played a thousand times, and went back to playing video games or watching TV, letting my mind wander to things that were in no way constructive or helpful. For this, I have to thank my wife, who is now living her dream. I was playing a really hard guitar song on Rock Band 3 and said “I wish I had the real guitar controller… or even better that I was just playing guitar right now.” She looked at me as though I had been replaced by some kind of 50s sci-fi monster and said “then… go play your guitar.”

She has said something similar to me for years, but sitting at her computer with her tablet in her lap working on a commission made me realize: 1) Holy shit. 2) I’m an idiot.

So most importantly, I got really honest with myself about my life. No, it’s not going to be easy. No, it’s not going to be cheap, and it’s not going to be fast. But I’m going to start working on myself a lot more aggressively. I’m going to start being the man I want to be, one step at a time. And the most important step, right here and right now, is being absolutely, breathtakingly, irrevocably honest with myself and everyone else. Da Vinci had Pope Alexander VI’s son, and various other patrons to pay his bills as he created everything he ever wanted to. Charles Ives sold insurance to finance his career and support his family. I can’t compare myself to such legends of the things I respect, at least not if I’m being honest with myself.

But maybe in a few years, I can say I even came close to that. Being honest, I may fail. I may end up fading into obscurity like everyone else who wanted to make their mark on the world. But being honest: I’m okay with that. At the very very least, I’m going to try. I’m never going to stop trying. Being honest with myself, I may not always rise to that challenge, I may have to put off this nebulous dream for years at a time. But living with purpose is a full-time job, and sometimes you need weekends off.

Graceful Degradation or Progressive Enhancement?

Feb 26 2010

The HTML5 Spec is trudging slowly (yet inevitably) closer to completion, and CSS Level Three is being picked up by Mozilla at a rate almost equal to their Webkit equivalents. What does this mean for those of us who make a living building websites?

Simply put, we’re not all getting the same internet. As argued by the anonymous and yet ingenious Do Websites Need to Look Exactly the Same in Every Brower.com, different browsers will provide a different appearance of the same overall content and presentation. So we have essentially three choices.

Screw the new features and stick with what we know will work across the board.
Use the new features and ignore the old browsers, people on IE will just have to deal.
Make a site that works on both new and old browsers, but provides a different experience for each

So Where Does That Leave Us?

A majority of the web is moving towards option number 3. This honestly makes a lot of sense, as CSS 3.0 may be delayed even further, and IE9 may just decide to only support a small margin of the features. HTML5, for that matter, might take another decade to be fully supported by any one browser, let alone all of the market share. So now the question becomes the following: do we design for older browsers and then add neat features that are supported by the newer browsers, or do we design for newer browsers and then let an “acceptable” level of compatibility fall to the older browsers?

This defines the current argument of the web community, and there are official terms for each side. A degradable design is one that has fallback features for older browsers, but is truly optimized for newer and more standards compliant ones. A progressively enhanced design is one that is built with the less standards-compliant browsers in mind, and then steadily adds non-essential components for newer browsers.

In My Opinion, The Argument Is Perhaps A Bit Premature.

As a musician and sound engineer of some years of experience, I was constantly asked to “mix for headphones” or “master this so it sounds good in the car.” Just like most musicians who have pulled a majority of their hair out in the vain attempt for the “perfect mix,” I can tell you with some level of authority (and with a hairline to prove it) that this holy grail does not exist.

That’s not to say we can’t strive to achieve it, but one way or the other, our product is going to inevitably be slightly different on some viewing platforms. Is this the end of the world? To quote my earlier resource… NO!

So What Do We Do?

For now, the internet is creeping towards a new standard, and it has (widely accepted) old standards upon which to fall back. I say that as of today (February 22, 2010), if your target audience is the majority of the internet (something like 62% of the market share), design your sites to be html 4.01 and xhtml 1.0 compliant, and use as much CSS Level 2 as possible. This doesn’t mean be lazy, or slack on the layout and semantic value of your content. it means simply do your best to make a decent, accessible website. Then, you can add text shadows, gradients, rgba colors, whatever you like to your stylesheets so that more technically savvy users are rewarded with a slightly “improved” look. Think of it as making your site accessible to people who haven’t upgraded. Treat IE6 like a screen-reader. If people can see it, and it makes sense, it’s good enough.

That being said, if you’re someone who only plans to impress the users who are on the latest version of their Gecko, Webkit, or Opera-powered browser, go full hog as soon as possible with the new features. The sooner we push the browser manufacturers and users, the sooner they’ll be forced to adopt the latest technology. Besides that, if you end up learning all of the advanced functionality of the new web technology a month late, you’ll be a year behind the curve in developing it.

So Wait, You’re Saying Both Sides Of The Argument Are Right?

Like anything else, I think this argument needs a healthy dose of perspective. Who are your clients? Who is their target audience? Who are you trying to impress?

If you don’t know the answers to these questions, find out as soon as possible! This is your due diligence as a digital marketer. If you’re hiring an interactive agency to do this for you, get this information to them as soon as possible. Give them your branding guides, your technical requirements, and any documentation that you have lying around immediately. Then, you can decide together whether your site should degrade gracefully across the spectrum of technology, or whether it should be enhanced progressively in order to provide the highest level of accessibility.