Recently I received an unsolicited request for help from someone I’ve never met and whom I do not want to expose. The sender confused me for a Twitter employee, I guess. Here’s the request (paraphrased):

Greetings,

How can I use Twitter for business messaging but block people from sending us tweets back? I’d like to guarantee the public can see our tweets but we don’t see theirs.

I pointed the sender to twitter.com for assistance but could not let this go without sharing the hilarity of the request.

First, I use Twitter but have no relationship with Twitter. I have no idea why someone would think I do.

Second, the request is so totally against social media principles as to be a poster child of “Does Not Get the Concept.”

Twitter and other social networking venues are about sharing, co-communicating (emphasizing the two-way nature because too many one sided message givers call themselves and their function “communicating”) and community. The idea that someone could disable others’ ability to write tweets to one’s own account is, well, preposterous.

However, I firmly believe that many businesses and other communicators would actually prefer a one-way messaging system that would guarantee public consumption but prevent public response. Just so happens this misguided inquirer is more blunt and honest about his or her intent. Others actually use Twitter and Facebook, etc., in this same manner: output only, no input acknowledged.

Most other media are exactly this kind of one-sided “unication” (my term for these “communicators”) that push out messages but have no mechanism or intent for dialogue. Radio, TV, infomercials, print media, banner ads, etc., all favor the message sender and diminish or prevent the reciprocal exchange of information and feedback. Even email has become a vehicle for unication; consider “noreply@example.com”.

I believe one reason for the uptake and success of social networking is the inherent ability to engage in two-way conversations.

It will be interesting to see how Twitter changes with the introduction of Promoted Tweets — that is, in stream paid advertisements. Will this push Twitter to become yet another unication platform? Will Twitter be able to retain the “social” aspect of its service post Promoted Tweets? I hope so.

There’s probably a better way to do this. (If so, leave a comment!) However, I run a debian server for a small task and use the default MTA Exim for mail delivery of status updates to my main mail accounts. Usually I run Postfix or Sendmail. I even have to support Qmail on occasion. However Exim is unfamiliar territory for me — I’m an Exim n00b.

Recently I decided I wanted to remove the “frozen” messages seen by running the mailq command. For example:


# mailq
48h 1.6K 1NUQLB-0007BC-QL <> *** frozen ***
user@domain.tld

48h 1.6K 1NUQO9-0007CW-1u <> *** frozen ***
user@domain.tld

48h 1.6K 1NUQOG-0007CY-1f <> *** frozen ***
user@domain.tld

48h 1.6K 1NUQOG-0007CZ-1v <> *** frozen ***
user@domain.tld

48h 1.6K 1NUQP4-0007DH-Af <> *** frozen ***
user@domain.tld


I know they’ll be purged eventually, I just wanted to clear the queue for some quick testing I was performing. The documentation I found on-line mentioned running “exim -Mrm <id>” for each message id in the mailq output. Being lazy, I don’t want to run that each time — I want a simple one-line command to purge frozen messages. So, I wrote the following BASH script that parses the mailq output and executes the command for each message id:

for i in `mailq | awk '$6 ~ /^frozen$/ {print $3}'`; do exim -Mrm $i; done

Worked like a charm! Message queue emptied of frozen messages.

This is a hard post to write since it is absolutely embarrassing to me and may cause me to lose serious geek-cred. However, I hope this is a useful mea culpa for others’ benefit. This morning I spent a while revising a script that accepted external input which I borrowed (and modified) from someone else. The original script used the external input safely – my modifications made it a true security nightmare. Basically, the script took content from the Twitter API and rerouted it through Jabber (XMPP) protocol. My change also sent the content through email.

If I had used the built-in SMTP features of ruby this post would have been unnecessary. However, as I was in a hurry (the excuse of a scoundrel) I simply piped the output through BASH to the mail command via ruby’s system() function. Also, I took no precautions to escape the external input from being executed by BASH itself. System admins and top-notch developers are now crying with laughter.

Why did I do this? Besides being an idiot, I did this because this script was a “one-off” and not something I was paid to develop. I needed a feature and, in the spirit of a developer (not a system administrator or security professional), I looked for the path of least resistance. With the system() call I could use my knowledge of the Linux command-line to accomplish my task. Since the original script was written in ruby, a language I have not used much at all, previously, I was able to add my needed feature in minutes. A couple more minutes would be all that was necessary to find out how to use ruby itself to accomplish this result. Laziness won over thoroughness and correctness. Result: a *huge* security hole.

Here’s what prompted me to invest in learning to add this feature in the Right Way™: a friend of mine who is an überGeek realized that, based on a clue I gave out about my script’s performance, he could have his way with my server via Twitter. I was Twitter-pwn’ed. Here’s what he sent me (knowing I watch my tweets via this script):

@rjamestaylor $(touch /tmp/test)

@rjamestaylor $(ls -la /tmp/test)

After he sent these two tweets my script sent me an email with this:

-rw-r--r-- 1 root root 0 Dec 27 03:39 /tmp/test

Yes. BASH executed his tweet (!) and as ROOT! So, if someone had tweeted “@rjamestaylor $(rm -rf /)” my server would be a barren husk of hardware. Worse, someone could have used my script — via Twitter! — to setup privileged user accounts, send spam (though small), attack other systems, etc. Why root (or as my geek friends would ask, WHY IN GOD’S NAME “root”?)? Simple: I was in a hurry. Stupid.

Lesson: spending the little bit of extra time necessary to add features without violating security protocols is always worth it. Sure, I could have just bought some CodeOffsets to assuage my guilt, but it was actually better to (1) shut down my script until (2) I fixed the security issues.

Turns out adding net::smtp to a script in ruby is painless. Now I have my feature and no exciting side-effects from rushed, careless development. In fact, most security holes I encounter in others’ code is due to the same thing: rush to add features without a review of implications. Anytime one takes external input from a user or the Internet (or Twitter) one must sanitize the input from unintended effects. This goes for SQL injection (parameters, people!) as well as simple command expansion attacks, among many other possible vectors.

So, go ahead and tweet BASH-expandable tweets to me, my code is not just offset, it’s fixed.

Twitter List support has been out for a while now but the impact of this new feature may not be readily apparent to most Twitter users. Yet, Lists have already changed Twitter in dramatic ways that you really need to know about.

Twitter Lists are user-created groupings of other Twitter accounts. This is somewhat like the client-side groups provided by TweetDeck, Seesmic Desktop, FriendFeed and others but since it is built into Twitter itself the impact is far-reaching. The biggest difference between client-side groups and Twitter Lists is that lists are public. User can “follow” lists others set up, for example.

Here’s a quick rundown of how Twitter Lists work. A logged in user can create up to 20 Lists containing 500 users each. The name of the List is up to the user and usually suggests the common trait tying the users together in the mind of the List creator. The list creator is also able to provide a description of the list created. The list can be public (default) or private (seen only by the list creator). However, users added to a list can see which list they are on regardless of the public/private setting of the list itself. That means users on a private list know who has listed them and knows the name of the list that they are on. [ed: Thanks to several people who have corrected me - I agree, private lists are private to the list curator and are not seen by those listed. My apologies!]

@rjamestaylor home page with count of Lists

When I log into Twitter I can see that I’m currently listed on 89 Lists (see above). I can also see the Lists to which I’m subscribed on the right most column of my Twitter home page.

Lists @rjamestaylor created or follows

Let’s create a list! The screencast below demonstrates creating a new list, adding users to the list, viewing the list and deleting the list. (Tip – watch in full screen mode)

The most obvious benefit of using Twitter Lists is to reduce the clutter in your reviewing of users’ tweets. By segmenting users into lists the “noise” of all the twitter users you follow is drastically reduced. But the game changing value of Twitter Lists is more subtle: following lists others make of interesting users.

Making lists and checking them twice

Since anyone can make lists of any (non-blocking) Twitter users and share these lists with the Twitter public a new way of discovering interesting users has come to Twitter. The impact is this: those users with special knowledge of other Twitter users can group otherwise unidentifiable users into easy to find categories. For example, if you’re interested in venture capitalists, you may have tried to find those VCs on Twitter you know of or can find through searching Twitter profiles for their own self-description as a VC. Yet, you may miss some key VCs without special knowledge. Twitter @Scobleizer probably knows more VCs on Twitter than most and he has faithfully created a list of VCs on Twitter: @Scobleizer/venture-capitalists. Now you can easily follow the thoughts, activities and actions (including locations) of the top venture capitalists on Twitter. Needless to say, finding trends among VCs is significantly easier for all of us due to Twitter Lists.

Let’s take this a step further. Say you are interested in a particular company for reasons such as news reporting, competitive intelligence, investment research, employment research, or even to follow the tweets of your own employees. Previously, finding Twitter users from particular companies relied on the self-reporting of each user. Now those with special knowledge of a company’s staff can make their own lists. So an employee of a company may not realize they have been listed by others as part of their employer’s Twitter users. Impact? Your personal intention to separate business from personal life on Twitter is compromised. More over, those companies that make their own list of employees who tweet make it very easy for the public to learn about the culture, sayings and doings of their employees. By having a public list of employees whether self-made or made by others with special insight, researching a company has been simplified. PR departments may think they control the public perception of the internal nature of their companies but Twitter Lists has busted this wide open.

Previously a Twitter user may have been followed by a handful of others. Such a user may have been careful not to related themselves to their employer but by being listed as an employee of their company they have become a de facto spokesperson on their company’s behalf. This is why I believe Twitter Lists have changed everything about Twitter.

For example, say a company is going into a quiet period required by the SEC. Because of Twitter Lists employees that believe they are speaking independently and privately now may give critical information to investors through tweets like, “Wow – just met our new owners – great company! Can’t wait until I can talk about this!” As part of a List of employees this may be an improper disclosure. This is an extreme example, sure, but a real one that could have employment and legal consequences for the employee.

Example two: an airline is facing inquiries from the FAA over possible maintenance violations. An employee not knowingly aggregated with his employer due to being listed on “Airline-X-engineers” list tweets, “finally the chickens are coming home to roost. Shortcuts we’ve been forced to make are getting exposed.” Oops.

Example three: a utility company suffers a public-affecting outage. Many people are impacted. Phone lines are jammed, customers are hurting. An unwittingly listed employee tweets, “Nothing to do at work so we’re catching up on World of Warcraft.” How do you think customers would react when the PR department releases a statement that everyone is working hard to recover power to each customer? Perhaps the employee works in accounting for the utility and their department really is at a point when the workers are able to catch a break. Think customers will understand that an accountant isn’t going to be manning the cherry pickers? Ouch!

Rethinking Twitter in light of Lists

Twitter has changed, dramatically, with the introduction of Lists. As a long time Twitter user I see that the former semi-anonymity associated with tweeting was seriously compromised by Twitter Search and practically eliminated due to Twitter Lists. I believe we all need to rethink some previously held assumptions about Twitter.

• Should I tweet at all?

Previously I encouraged everyone to use Twitter. However now more than ever we must realize that tweeting can affect our personal and professional lives. Those with any concern that their tweets could jeopardize their livelihood should seriously consider not tweeting at all.

Those who continue to tweet need to remember that much is on the line with each tweet. Deleting a tweet is not guaranteed to remove it from the Internet (see: Once Tweeted, Never Deleted). Remembering to ask, “is this hurtful? Is this helpful?” before tweeting is necessary now more than ever.

• I’m listed as an employee of my employer, what should I do differently?

First, make sure to carefully follow SEC and company policies with each tweet. Not sure what those rules are? Better read step one above and find out before continuing to tweet.

Second, before tweeting make sure that your company’s customers are not suffering on Twitter. How? Search for your company’s name or brands on Twitter using Twitter search. If your customers are suffering try to reach out to help them or consider not tweeting until the issues are resolved.

Third, remember that your company will be judged in aggregate through the tweets of listed company employees. Proud that your company provides expertise to customers related to technology? How about including relevant links every once in a while, or offering up a simple tip related to some area of your own expertise? That will positively impact not only the public perception of your company but help show you personally are a contributing member of the company to others. Conversely, if your company sells vegan products as its main product offering, tweeting about hunting trips and company outings that include baby backed ribs may have a negative impact on the company and your employment.

• If I’m listed without wanting to be, what can I do?

Above I show how to tell whether or not you appear on any lists — check that frequently if you are concerned about which lists you appear. Currently Twitter users can add anyone that is not blocking them to a public or private list of their own choosing. So if you do not want to be on a particular list I recommend first trying to contact the list curator to have you removed. If that does not work, the more extreme step of blocking the list maker will remove you from any list you on which you do not wish to appear. Blocking is considered extreme and even rude, so use this as a last resort.

• I want to use this new, powerful research tool – how do I get started?

Due to my concern over the average Twitter user I’ve focused on the worrisome aspects of Lists in this blog post. I care that people not be surprised about the changes to Twitter due to Lists and that’s my main focus here. However, Lists are powerful for reading as well. The best place I’ve found to begin exploring Lists is a third-party website specializing in Lists: listorious.com. On Listorious the top lists and list makers are curated and made searchable. Spend some time there to find lists that interest you. Once you find interesting lists and follow them think of the lists you would like to create and start making your own. Do be polite and sensitive to others that you list. Blocking may be considered rude but adding people to a list they do not want to be on can be hazardous to them, personally.

Thanks everyone for reading along. I hope you’ve enjoyed reading these
tips as much as I’ve enjoyed writing them.

For the last installment, I wanted to share a very cool feature in VIM
that I am just beginning to learn how to use. VIM provides support for
Perl, Python and Ruby so that you can use these languages to create
functions in VIM.

I’m most familiar with Python myself, so here are a couple of examples.
In VIM, try the following:

:python print "Hello world"

You’ll see ‘Hello world’ show up in the status line at the bottom of the
screen. Cool, but not all that handy (though you can use this as a
quick calculator, e.g. :py print 256 * 8 ).

To actually get Python to do something interesting with the contents of
your editor, you can define a VIM function that uses Python to do the
heavy lifting.

Here’s how:
:function! PySort()
python << EOF
import vim
b = vim.buffers[0]
x = b[:]
x.sort()
b[:] = x
EOF
endfunction

The :function! line begins a function definition. VIM has its own
internal scripting language, which is swell and all, but if you already
have familiarity with one of the other supported languages, you can use
that language to get a jump start on seriously tricking out VIM.

The line:

python << EOF

tells VIM that we’re defining a block of Python code. The block will
end with the line “EOF”. The enclosed lines are pure Python.

First, we import the vim module:

import vim

Now we have access to that module’s components, like the buffers[]
list. Just like in normal Python, buffers[] is zero-indexed, so
buffers[0] is the first buffer.

Next, we copy the contents of b as a list into the variable x:

x = b[:]

Then we sort that list alphabetically:

x.sort()

Then we replace the current buffer with the sorted context of x:

b[:] = x

And voila – you can now sort the current buffer alphabetically by
calling the PySort() function:

:call PySort()

This is just a trivial example, but hopefully it gives you some ideas of
what can be done with Python (or Perl, or Ruby) inside of VIM.

To get more information about using scripts in VIM, try the following
help commands in VIM:

:help python-vim
:help perl
:help perl-using
:help ruby-vim

Thank you all for reading. Anyone interested in exploring any of the
topics discussed in this or previous tips should feel free to contact me.

Happy VIMming!

- David Roth

VIM’s got a lot of helpful commands that start with g. Not sure why g,
of all letters, but there you are.

One that I find really helpful when running through Apache
configurations is gf. If you put your cursor over the path to a file
and type:

gf

VIM will open up the file that’s under your cursor. So, given a vhost
like the following:

<VirtualHost *:80>
DocumentRoot /var/www/html
ServerName example.com
</VirtualHost>

If you put your cursor over /var/www/html and type gf, VIM will open up
a directory listing of that directory. Or, if you’ve piped the output
of httpd -S to a file and you put your cursor over the path to a
particular virtual host’s configuration file, you can open it up in the
same way. Note that if you’ve updated the current file and it hasn’t
been written to disk yet, VIM will complain that the current file hasn’t
been saved yet and gf will fail.

Another handy use of g is when you’re editing a file with wrapped
lines. Ever find yourself on a line that is wrapped, and you want to
make a change that’s right below the cursor, so you type j or hit the
down arrow, but the cursor skips right past what you want and goes to
the next line? Don’t you just HATE that? Well, if you type gj or
g<down arrow> instead, the cursor will move according to displayed
lines, not logical lines. Much better.

Finally, my favorite stupid VIM trick. g? will ROT-13 text. Use the
ggVG sequence to highlight your entire file. Then type:

g?

And presto – every alphabetic character is shifted 13 places to the
right in the alphabet. Quick and easy obfuscation.

Happy VIMming!

- David Roth

We went over creating VIM macros in a previous tip. If you save your
macros to a file, you can have VIM run the macro against a file,
effectively creating a batch mode for executing VIM macros.

Let’s say that, for whatever reason, you want to convert a file to all
lower case. Create a file called lower.vim and put in the following lines:

ggVGu
:wq!

Make sure there are carriage returns at the end of each line. Those
commands will jump to the first line (gg), turn on visual highlighting
mode (V), then jump to the last line (G). The ‘u‘ command at the end of
the first line will translate all highlighted characters to lower case.
Then the next line writes and quits the file.

Now you can apply this script to any file with:

vim -s lower.vim /path/to/file

Maybe you want to translate all your XML files under the current
directory to lower case. In that case, you can combine VIM with ‘find’
to do some quick batch processing:

find . -type f -name \*.xml -exec vim -s lower.vim {} \;

Note that when you use VIM in this way, VIM does not operate in the
background. An instance of VIM is started for each file being
processed, and the commands in the script file are run like a macro. If
your script file makes a lot of complicated changes, this can be really
amusing to watch. I run this on my computer at home instead of having a
fireplace.

Happy VIMming!

- David Roth

VIM ships with a number of plugins, which are scripts that enhance VIM’s
basic functionality. You’ll find the plugins that shipped with your
copy of VIM in $VIMRUNTIME/plugin.

(Sub-tip: don’t know where VIM is installed? In vim, type “:echo
$VIMRUNTIME“.)

We talked about syntax highlighting in an earlier tip. Syntax
highlighting is great and it makes code more readable, but if you take
your text file that’s all fancy in VIM and publish it on a web page
somewhere, it goes back to dreary black text.

But it doesn’t have to. With a syntax-highlighted file open, type:

ggVG

(that highlights the entire file)

:TOhtml

(that creates an HTML document of your highlighted file)

Cool, huh?

Another fun pair of plugins are gzip.vim and tarPlugin.vim. These are
great because you don’t have to do anything to activate them – just use
VIM to open a .gz or .tar file (or, for that matter, a .tar.gz file).
VIM will uncompress your gzipped file on the fly and re-compress it when
you’re done editing it. If you’re opening up a .tar file, VIM will give
you a list of the files in the tarball.

Happy VIMming!

- David Roth

VIM has a bunch of built-in event handlers which can be used to automate
certain tasks. These are known as autocommands. You can define actions
that get executed whenever one of these autocommands is fired to do
things like read in a template file or write a note to a log file. In
today’s tip, I’m going to show you how to use template files for certain
file types.

Let’s say you write a lot of HTML code, and you always find yourself
creating the following structure:

<HTML>
<HEAD><TITLE></TITLE></HEAD>
<BODY>

</BODY>
</HTML>

Instead of having to type that every single time you create a new .html
file, you can write that structure into a template file. In your home
directory, there should be a .vim directory. If there isn’t go ahead
and create one, then create a directory called templates and write your
template file under the name html.tpl.

Now, add the following line to your .vimrc:

:autocmd BufNewFile *.html 0r ~/.vim/templates/html.tpl

That line tells VIM that you’re defining a new autocommand that is to be
executed when the BufNewFile action is triggered. BufNewFile gets
triggered every time a new file is created in vim. In this case, if the
new file’s name matches the pattern “*.html”, it’s going to read in the
contents of ~/.vim/templates.html.tpl. So now, if you type:

vim brandnewfile.html

VIM is going to create the new file and read in the contents of your
HTML template.

One note about this tip – I have no idea what the 0 in “0r” is for. If
anyone knows why the zero is needed, please let me know.

Happy VIMming!

- David Roth

This is why ninjas are scared of VIM – it allows you to see that which
cannot be seen.  This is a trick that I learned when editing a lot of
Python code in a massive project.  As you may or may not know,
in Python, whitespace is syntactically significant.  Indented lines are
used to indicate the bodies of functions, classes, and loops.  Problems
can arise when bothspaces and tabs are used for indentation, because
Python doesn’t necessarily consider eight spaces to be the equivalent
of two tabs, even though they may look the same to a human eye.

So when you have a file open in VIM and you want to view non-printing
characters, you can turn on list mode to see non-printing characters:

:se list

With list mode enabled, tabs will appear as ^I, carriage returns will
appear as ^M, etc.  To turn list mode off, type:

:se nolist

If you see a character and you’re not sure what it is, you can use
another couple of tricks to view the ASCII value of the character.  Put
your cursor on top of it and type:

ga

(Think “get ASCII”).  If you’re editing a UTF-8 file, the command is:

g8

Then you can look up the value at http://www.asciitable.com/ or
http://www.utf8-chartable.de/ to see what it is. (Unless you’ve memorized the ASCII and UTF-8 tables, in which case you’re probably an emacs user and not interested in these VIM tips anyway.)

Happy VIMming!

- David Roth