blackrat.org

In Part 1, I took you through the beginnings of the Fnord generator up to the point we could create Fnords using random words and optional parts of speech. This gave us a class Fnord which contained the following functions.

require "fnord_words.rb"  #Arrays of NOUNS, ADJECTIVES, PLACES etc.
class Fnord
 def self.true?(chance) (chance==0 or rand(chance)<1) end
 def self.build(string_array,chance=0) true?(chance) ? string_array[rand(string_array.length)] : "" end
 def self.in_place(chance=0) true?(chance) ? "in #{build(PLACES)}" : "" end
 def self.adjective(chance=0) build(ADJECTIVES, chance) end
 def self.name(chance=0) build(NAMES, chance) end
 def self.place(chance=0) build(PLACES, chance) end
 def self.preposition(chance=0) build(PREPOSITIONS, chance) end
 def self.action(chance=0) build(ACTIONS, chance) end
 def self.pronoun(chance=0) build(PRONOUNS, chance) end
 def self.intro(chance=0) build(INTROS, chance) end
 def self.noun(chance=0) build(NOUNS, chance) end
end

I moved the word lists into their own file “fnord_words.rb”. Since these are separately generated and updated by SJ Games, it made sense to have them as separate files. I thought of writing a quick Perl->Ruby conversion to allow for the files to be dropped in, but since manually updating requires making only small changes to the file, I decided to leave this as an exercise for a later date.
I was using the normal

msg=case rand(14) #Return generated Fnord as a string
when 0: "The #{adjective(2)} #{noun} #{in_place(5)} is #{adjective}."
when 1: "#{name} #{action} the #{adjective} #{noun} and the #{adjective} #{noun}."
when 2: "The #{noun} from #{place} will go to #{place}."
when 3: "#{name} must take the #{adjective} #{noun} from #{place}."
when 4: "#{place} is #{adjective} and the #{noun} is #{adjective}."
.
.
.
when 13: "A #{noun} from #{place} #{action} the #{adjective(2)} #{adjective(5)} #{noun}."
end

and I figured I could move all of the data including the SENTENCES into a template to make it even easier to update. This required two things to happen.

1. All of the parts of speech required are embedded in the string.
2. The parts of speech can only be evaluated at runtime when the method is called.

We’d already achieved the first, and in order to have the second all that was needed was to move the SENTENCE strings into single quotes, such as

'#{intro(5)} the #{adjective(2)} #{adjective(2)} #{noun} #{action} the #{adjective(2)} #{adjective(2)} #{noun} #{in_place(2)}.'

In order to execute this later, you use the eval function to perform substitutions. Ignore normalize for now, it’s only to remove extra spaces and fixup capitalization.

def self.sentence(chance=0)
  normalize(eval('"'+build(SENTENCES,chance)+'"'))
end

private
def self.normalize(msg)
  while msg.include?("  ")
    msg.gsub!(/  /," ")
  end
  msg.gsub!(/^ /,"")
  msg.gsub!(/ ./,".")
  msg.gsub!(/[s^]([aA])s([aeiouhy])/,' 1n 2')
  msg[0]=msg[0,1].upcase
  while msg[/([^A-Z][.!?:])s+([a-z])/]
    msg[/([^A-Z][.!?:])s+([a-z])/]="#{$1} #{$2.upcase}"
  end
  msg
end

and voila. Sentences can be constructed by calling

print Fnord.sentence

We are still missing the word lists. You can grab the latest updated Perl version from SJ Games, or, if you are feeling really lazy, download my updated Ruby code from my website and grab my personalized Rubyised word lists from here. They aren’t strictly the Fnords word list, so you may prefer the SJ Games ones, but they are in the correct format.
If you’ve gone down the Perl wordlist route, you won’t have found the SENTENCES array, which you need for my code to work. You can either modify my oroginal code, or use the expanded versions below.

#Note. Parts of speech are coded in the main application and MUST NOT be expanded here. Hence only ' rather then " can be used.

SENTENCES=[
	'#{intro(5)} #{name} #{action} #{name} and #{pronoun} #{adjective(2)} #{adjective(2)} #{noun}.',
	'#{intro(5)} #{name} #{action} the #{adjective(2)} #{adjective(2)} #{noun} and the #{adjective(2)} #{adjective(2)} #{noun} #{in_place(2)}.',
	'#{intro(5)} #{name} #{action} the #{adjective(2)} #{adjective(2)} #{noun} of #{place}.',
	'#{intro(5)} #{name} #{preposition} #{place} and #{action} the #{adjective(2)} #{adjective(2)} #{noun}.',
	'#{intro(5)} #{name} #{preposition} #{place} for the #{adjective(2)} #{adjective} #{noun}.',
	'#{intro(5)} #{name} is the #{adjective(2)} #{adjective(2)} #{noun}; #{name} #{preposition} #{place}.',
	'#{intro(5)} #{name} must take the #{adjective(2)} #{adjective(2)} #{noun} from #{place}.',
	'#{intro(5)} #{name} takes #{pronoun} #{adjective(2)} #{adjective(2)} #{noun} and #{preposition} #{place}.',
	'#{intro(5)} #{place} is #{adjective} and the #{noun} is #{adjective}.',
	'#{intro(5)} a #{adjective(2)} #{adjective(2)} #{noun} from #{place} #{action} the #{adjective(2)} #{adjective(2)} #{noun}.',
	'#{intro(5)} the #{adjective(2)} #{adjective(2)} #{noun} #{action} the #{adjective(2)} #{adjective(2)} #{noun} #{in_place(2)}.',
	'#{intro(5)} the #{adjective(2)} #{adjective(2)} #{noun} #{in_place(2)} is #{adjective}.',
	'#{intro(5)} the #{adjective(2)} #{adjective(2)} #{noun} from #{place} will go to #{place}.',
	'#{intro(5)} you must meet #{name} at #{place} and get the #{adjective(2)} #{adjective(2)} #{noun}.'
]

Enjoy

I recently came across the Steve Jackson Games Fnords program and thought that there should be a really cool and easy way to generate this type of sentence using Ruby. The basic priciple is that you take arrays of NOUNS, PLACES, VERBS, and other parts of speech and generate syntacically correct nonsensical sentences. A little bit like a truncated form of madlibs.

Ruby embedded code in strings was something that I’d played around with, and I thought that it should be relatively easy to base sentence construction using methods embedded in the strings. So basically

"#{name} is a #{adjective} #{noun}."

could be run through the generator and create “Mac is a brown dog”, “Sigmund Freud is a coherent lightbulb”, type of sentences. Fairly basic stuff with NAME, ADJECTIVE and NOUN being arrays of appropriate words and the method, for example, of self.name being defined as

def self.name NAME[rand(NAME.length)] end

If you’ve taken a look at the other Fnords programs at SJ Games (or even my early one which is now downloadable from there), you will have noticed that the sentences themselves have a random chance of having some parts of speech included. It was this random element that first attracted me to solving the problem using Ruby.
Most code on the site uses logic of the following form.

msg="The"                           # The
if rand(2) == 0	               	    # adjective - (50% chance)
  msg+=adjecive
end
msg+=noun                          # noun
if rand(5) == 0                      # in place  - (20% chance)
  msg+="in #{place}"
end
msg+="is #{adjective}."         # is adjective.

That wasn’t DRY enough for me, and too much code when a little would do the same thing. Moving the (rand()) into the actual function that defined the part of speech and making the return from the call optional would do almost the same thing. Worst case I would have to fix up spaces at the end. (I actually decided to do space fixup as a final pass, since it meant I could just type the sentences spaced normally, which makes it easier to maintain.)
Ok so with the method for noun now as

def self.noun(chance=0)
  if (rand(chance)<1) then
     NOUN[rand(NOUN.length)]
  else
     ""
  end
end 

The way the Ruby rand works, passing it 0 gives a random number between 0 and 1 and passing a positive integer greater than 0 generates a random integer, so with this code rand(0)<1 is always true, so the next iteration changed this to

if (chance==0 or rand(chance)<1)

Finally to DRY it up even more, I removed the randomness into two separate methods and switched to the “? :” form of “if then”.

def self.true?(chance)
  (chance==0 or rand(chance)<1)
end
def self.build(string_array,chance=0)
  true?(chance) ? string_array[rand(string_array.length)] : ""
end

which meant that my noun method could now simplify again to

def self.noun(chance=0)
  build(NOUNS, chance)
end

One special case was required to allow for “in place” as optional rather than just “place”.

def self.in_place(chance=0)
  true?(chance) ? "in #{build(PLACES)}" : ""
end

Now I could create strings using my desired form.

msg="The #{adjective(2)} #{noun} #{in_place(5)} is #{adjective}."

Part 2 will examine how to take this and create the fully functional fnords program.

Whilst creating a natural language parser, one of the things I was presented with was multiple merged dictionaries, which needed some processing. I was asked to supply a list of duplicated words back, and after trawling the web finding slow code, I decided that going the fast, but inefficient (in terms of space) was the way to go.
The object is to return an array of just those elements that are duplicated in as little time as possible, from a 50,000 word list. One sort and one lookup per element was what I came up with

array=["long","array","with","lots","and","lots","and","lots","of","array","duplicates","long","and","array"]
arr2=array.sort
arr3=[]
arr4=[]
arr2.each { |a| (arr3[-1]==a) ? (arr4[-1]!=a) ? arr4 << a : "" : arr3 << a }
arr3 => [”and”,”array”,”duplicates”,”long”,”lots”,”of”,”with”]
arr4 => [”and”,”array”,”long”,”lots”]

If you are sure that there are fewer duplicates in the returned array of just duplicated elements, you can remove the arr4 checks at the expense of a final .uniq! pass. i.e.

array=["long","array","with","lots","and","lots","and","lots","of","array","duplicates","long","and","array"]
arr2=array.sort
arr3=[]
arr4=[]
arr2.each { |a| (arr3[-1]==a) ? arr4 << a : arr3 << a }
arr4.uniq!

arr3 => [”and”,”array”,”duplicates”,”long”,”lots”,”of”,”with”]
arr4 => [”and”,”array”,”long”,”lots”]

Maintaining 4 separate blogs (2 project specific and intraneted, rather than external), doesn’t take any more time than 1, but it means that less gets added to this one than might otherwise be the case. However, one thing I can share is the work that’s been going on under the covers to get some Ajax code working on Blogger. (Yet another blog). This all came about when I had looked at some code and thought. Hmm. That would be so much easier to create, maintain and extend in Ruby.
I didn’t want the full blown features of Rails, but since I was experimenting with cutting down the code needed for yet another project, I thought that this would give me a good quick way of injecting Ajax code into a 3rd party site.
One of the rules of Ajax, is that you can’t go outside the domain for the call. So where an XMLHttpRequest.open(’GET’,'http://random.external.site.com/CGI_program’), would throw an error, you can do this by performing the call from an page on http://random.external.site.com.

For example, the header on Nofnords is an Ajaxified update from fnord.pqmf.com (a subdomain of this site), which has been embedded using an iframe which uses http://fnord.pqmf.com/fnord.html as the jumping off point. What this means is that the XMLHttpRequest comes from the same domain as the iframe’d page. Since this is a separate request, there is still a separate between the domains, so data isn’t accessible between the two pieces of content directly, but with some CSS, the appearance is (almost) seamless, and allows for some “personalized” customisation that would otherwise be difficult to achieve.