Utterance Understanding for the Unintelligent Let's start from the most specific pieces and work outward. 0. Table of Contents: Section 0: Table of Contents Section 1: Deep Syntatic Structure Section 2: Dialogue Act Section 3: Unnecessary Confusion: Key Words Section 4: Relation Section 5: TextPlan Section 6: Dialogue Act Types 1. A Deep Syntatic Structure (DSyntS, or DSS) - This is a sentence wrapped in a .xml or .dss file in a way that the realizer module understands. - .dss examples can be found in here: realpro-manual.pdf - The reason we use the realizer module is because it enforces the grammatically correctness of our sentences. - The realizer is where we give up any more nit-picky, specific control over our sentences (But then again, these can get frighteningly specific, as you'll soon find out!) - Inside this structure is also where we can mark different parameters that can control the selection of each utterance form compared to another, such as polarity and complexity. - Keep in mind that these are meant to be slight variations on structures that say the "same" thing - Values like verbosity, which change things like HOW MANY of these DSSes are used, are not used here - All "_"'s seen are replaced as spaces in the output. I keep them in the variables to help distinguish them, though. - You can have as many DSSes as you want, and having more of them offers much more variability. Analogy Corner: - For an analogy, think of the DSSes as specific legos, collected in a bin of the same kind of legos (the Dialogue Act of section 2) - Each of these specific legos offers slightly different shades of color, maybe with rounded corners, or flowers painted on the sides. - You can build a house (an utterance) with nothing but one kind of every lego. With lots of different kinds, you can have window legos, roof legos, rose bush legos, the sky's the limit! - With lots of different kinds of legos, your house may come out nice, but you can only make that same house. Over and over. Like one of those creepy mass-produced, cookie-cutter neighborhoods. Boooooring. - Sure, you may move the garden's plants around, add an extra story, but humans are very keen on picking up similarities. This just won't do! - The more variation you have within a single kind of lego, the more personalized and interesting you can make every house on your lego street! Example 1: "SELECTION_X2 is the best restaurant of its kind" - As you can see here, it is a strongly positive statement (polarity 0.9) that is fairly complex (complexity 0.7) - These two attributes guide Personage to choose this utterance as a way to fullfill step 2 that has the personality we desire - We can add other attribute(s) related to the structure of the utterance to help us narrow down the pool of posibilities, if we desire Example 2: "SELECTION_X2 is not as good as the others." - From "assert-reco-worst", the discouraging recommendation - Here you can see the polarity is still high, because the negative utterance is gently delivered - You can also see a negation, combinging the "neg" form of "be" (is not) and the lexeme "as_good_as" leads to the id "as-bad-as" Example 3: "There could be worse restaurants." - Another example of "assert-reco-best", this time of a lower polarity (a much more pessimistic recommendation) - Notice that "complexity" is missing: all attributes are not required - This is the only option in the file that does not have a "SELECTION_X2" variable as a restaurant name. Example 4.1: "SELECTION_X2 is MOD_X." Example 4.2: "SELECTION_X2's price is MOD_X dollar." - These are the only two options with which to recommend based on a price - As you can see, they take very different input: One takes an adjective, the other, a number (also, more than 1 variable as input!) - All restaurant price values get converted to both types, so either can be used, providing another method/idea for variability - Some of the other structures, like saying the type of cuisine, only have one structure - These lack id's because there is no need to distinguish them Example 5: "John often eats beans" DSYNTS: eat [ class:verb ] ( I John [ class:proper_noun ] II bean [ class:common_noun number:pl ] ATTR often [ class:adverb ] ) END: - Here is an example .dss file as found in the realizer user manual. - There are dss-to-xml scripts, and vice versa, in case this is an easier form to write/read. 2. Dialogue Act - Time to set the record straight about "dialogue acts"! - A dialogue act is the word used to describe a file of a single type of DSS. Analogy Corner: - A Dialogue act is one kind of building block, such as the 2x3 rectangle, a roof tile, or a flower lego - The "kind of" (or category of) block is represented in the previous analogy as the "bin" that holds all of the same type together. - However, in this bin, there are all the different colors and slight variations that we wish to define of the same type. - You can see how many different acts are a necessity: you cannot build a standard house without a brick type, a window type, a door type, a floor type, etc. Example 1: "assert-reco-best" (very big file) - the file containing all methods of positive recommendation toward a restaurant in general, a single dialogue act Example 2: "assert-reco-worst" (also very big file) - Negative recommendations are stored in a separate file, a different dialogue act - It could be argued that they should share the same file, but then we would need another attribute to separate the positive/negative assertions. - It's simpler just to let them be in different files. Example 3: "assert-reco-price" - Finally, a look inside the file. All it is, however, is a tag which groups our list of DSSes. Simple, not much to it. You can see why I didn't want to post the two previous ones? - You can imagine dialogue acts as pulling from different sections of the same database, using one act to describe one kind of entry (in this case, price; in other cases, a type of cuisine, or an address) - Where DSS differentiation is important to have utterance flexibility, having a wide range of dialogue acts will come naturally. - Every idea that we want expressed in our sample dialogue will have to be able to be represented in one of these dialog act files. Example 4: "assert-location" (Made by Colin) - The single DSS reads "I think PERSON is at LOCATION" - As we go on, we may want to differentiate with a different name from other similar acts related to locations - For example, "I think PERSON is at LOCATION" sounds a lot different from "I hate LOCATION", but both could be named "assert-location" - The naming convection, however, is our own (see next section!) 3. Unnecessary Confusion (at Least for SpyGen): Key Words - Personage (for restaurants) slipped in another level of categorization here that we do not need. - Because of its superfluous nature, I was struggling for a long time to understand its purpose. - This level of categorization exists in file and function names, but not actually in any content we will be keeping. - If you don't fully understand yet, don't worry! The next few sections are very closely tied, and seeing the examples in all of them may help. - "Key Words" is my name for them, since I can't find any official references to them anywhere File Names: - There are more file names than function names for this weird purpose, possibly because they weren't all used in actual testing. - The two most prominent key words are "recommend" and "compare" - The key word is found in the second section of the naming of the most recent restaurant dialogue acts (where the last marks the act (section 2), and the first marks the dialogue act type (section 6)): "assert-reco-best" "assert-reco-worst" "assert-reco-cuisine" - Recommendations "assert-com-cuisine" "assert-com-decor" "assert-com-food_quality" - Comparisons - Another key word found in the file names is "summary" - An important note, here, is that most of these files that have a mirror (such as "assert-reco-cuisine" and "assert-com-cuisine") are EXACTLY THE SAME! - Not all files have a mirror, though, so I assume that's why they are differentiated. - The key word used to open files is "com", so there needs to be a "com" copy of a "reco" file in order for "com" to use it. Function Names: - As stated above, in the example I've been tracing, I see only "recommend" and "compare" used (in TextPlanBuilder.java primarily) - These determine how the dialogue acts are used in the code Why?: - Personage does not use standard TextPlans (section 5). However, we will be changing it to use them again soon. - Instead, these Plans are auto-generated in functions based on the evidence found in file names. - My own speculation: - Because the domain is currently so specific, they did not need to make use of the different relations (section 4) available in Personage - Instead, "justify" maps to "recommend" and "contrast" to "compare" (where justify and contrast are two of a handful relations) - Because of this, the different TextPlans were easier to generate than to make many different files out of. What do you need to know?: - That these Key Words won't be necessary! Don't worry about them! - They are part of a naming convention that we won't need to use and will hopefully soon be gone. - We can even make a new naming convention in their stead, if we find we need to categorize the Dialogue Act files a little better. 4. Relations - While a kind of side-step in the specific-to-general structure going on here, relations are very helpful in understanding TextPlans (section 5). - As you can probably guess by the name, relations are how different dialogue types (section 2) are combined. - Depending on other parameters, they can be combined in one sentence, multiple sentences, or even compounded on each other. - There are tags for the dialogue types used in relations that you have probably heard before: nucleus and satellite. (N/S) - These tags specify the importance of the dialogue type, basically whether or not the information is core to the utterance. - Satellite data can be thought of as supporting evidence, usually removed by a personality that is not very verbose. - As far as I can tell, there are 3 categories to relations described in the code: "mono-nuclear", "bi-nuclear", and a special case of no-nuclear - *** Example code is seen inside of TextPlans. Because of this, getting examples from Personage will be more confusing/difficult as of now. *** Analogy Corner: - You can imagine relations as the little pegs and holes that are a part of all lego pieces. - They stick the different lego pieces together in all kinds of ways, between both the same and different kinds of legos. - Brick legos need "cement" to keep them attached to each other, but also other building materials to attach roof and insulation legos! Example 1: Mono-Nuclear "Justify" - Mono-nuclear seems to be the most common way to combine multiple ideas. - Colin, Ben, and I experimented with this form in Sparky, so I will take an example from there (a piece from the TextPlan spy.xml): Example 1.1: How Relations Combine Dialogue Acts - In this example, id 1 is "suggest-go" to Kresge, and id 2 is "assert-location" Villain in Kresge - Running this example combines them, using the satellite to justify the nucleus: "Go to Kresge because Villain is in Kresge" What exactly does the combining? - Relations simply specify an intent for how the user wants two ideas to be combined. - How this gets realized in the code is through Operations. - ??? Should Operations be its own section ??? - Operations (such as using "because", "since", and all cue words) define how the realizer should restructure a sentence into the desired meaning. - Operations are chosen at random until one succeeds or they all fail. - Each operation specifies which relations it can satisfy, and if it can, the utterance is realized and the combination succeeds - The "fall through" operation if they all fail is the Period, which simply states the dialogue acts as individual sentences, doing no structural combinations. Example 1.2: Compounding ??? Should Proposition have a section too ??? - Here is an expanded example from above, where a new id 3 has been added to 1 and 2. 3 reads "assert-location" Hero in Crown - You can see here that, because the nucleus is the same, Sparky can drop the second instantiation of the nucleus and combine the two satellite info - One output option here is "Go to Kresge since Villain is in Kresge and Hero is in Crown." - It is so far unknown to me what happens when you mix-and-match relations, or try to talk about different information in the same TextPlan. - Because of how those TextPlans are automatically generated, I can't experiment too much with this yet. - Other mono-nuclear relations include "Elaborate", "Concess", and "Restate". Example 2: Bi-Nuclear "Contrast" *** All of my tracing so far as been with the "justify" relation. I will need more time to elaborate on bi-nuclear relations work. *** - My assumption here is that bi-nuclear relations give equal importance to both piece of information, making sure neither are dropped. - Makes use of the "while", "but", "however" Operations. - The other bi-nuclear relation is "Joint". Example 3: No-Nuclear "Infer" - Let's look at the xml stuff from The mono-nuclear relation: - It is my understanding that all Dialogue Acts must be represented using a relation, even if they are actually related to nothing. - This can get tricky, so try and stay with me! - NS/SN refer to Nucleus and Satellite, and in some Operations, their ordering. - You can see that each "Proposition" () has an "ns" value. - The code representations of relations and nucleus/satellite data is different than the mental image I have been building so far. - In the code, propositions have relations. Propositions can loosely be thought of as dialogue acts. - However, where as two dialogue acts can be related by a single "justify" relation (that is, 1 act justifies another), each proposition has its own relation. - Why? Not sure. It could just be a result of the data structures used to represent all this. - Either way, the important part here is WHAT infer is for, and that is to represent satellite data. - A proposition that is a nucleus, and thus required (ID 1), is given the "justify" relation to make sure it is kept. - However, the satellite data that is optional (ID 2), is given the "infer" relation. - When calculating verbosity, the code determines how many (0 or more) "infer" pieces of information to provide. - There are no other no-nuclear relations. 5. TextPlan - Those wonderful, mysterious objects that haven't existed since the time of Sparky! - Up to this point, I have mentioned them many times, so you might have an idea of what they do. - As DSSes tell the realizer what choice of words you're using and how you want them used (verbs, pronouns, adjectives, etc), TextPlans tell the realizer your choice of ideas. - TextPlans can be seen as the next level up in abstraction when thinking about sentences. - *** Currently there are not many TextPlans that I can extract examples from, but they are one of the first things we will be changing about SpyGen. *** Analogy Corner: - TextPlans are your lego house blueprints. - These blueprints tell you how to stick your roof on top of your walls, your walls on top of the foundation, and the fence around your house (relations). - Includes bonus instructions on how you build a roof or fence too (propositions)! Example 1: Revisiting spy.xml - Some of this code may be familiar! Example 1.1: Simple Pair - The rstplan is where the relation business is defined - The rest of the speech plan is the definition of the propositions, where those variables you saw in the DSSes get defined. - Here's a peek into these dialogue acts ("suggest-go" and "assert-location") to see how those variables are used (they slip right in!): - You can begin to see why they are currently procedurally generated. - Because the structure is mostly the same, except a few variables, the entire TextPlan business was automated. - However, while we will have a lot of repeated structures, we will also want to make these plans much more complicated. - Besides variability, it is much easier from a testing standpoint to have the inputs be TextPlan files. Example 1.2: Slightly More Complex - Here is an example I played with to see how we could get compounded relations. The output for this was seen in section 4's examples. - You just slip in relations and propositions for those relations. - *** As I've previously mentioned, there is much yet to be discovered on how different relations can be combined, or different nuclei defined. - Once SpyGen uses TextPlans again, this area will become more rich. *** Example 2: An Auto-Generated "Justify" Plan - This is what is in the code, both the gutted skeleton (Variables are surrounded by *'s) and an example filled in: - The purpose of the still confuses me, since it's not seen in the simpler TextPlans I played with in Sparky. - *** Another section to experiment with to discover its purpose! *** - Nevertheless, this is a glimpse into what we need to replace. - These seem to also explode (exponentially?) compared to the size of the DSSes. There will be lots of these to make after the system is changed. 6. Dialogue Act Types - A brief look at the last category seen flying around, a tying of loose ends. - The first word seen in almost all of Sparky or Personage's DSS file names, such as "assert" - *** I have not seen (or at least, can't remember) where these words actually matter in the code, but they are another classifying tool *** Analogy Corner: - The category of building your lego house falls into! - Some examples could be a 1-story house, 2-story house, apartment, duplex, mansion, or even a business or hotel. Example 1: Colin's Ideas - Colin brainstormed a few we might find useful in his SCHEMA file: inform: "There are three unvisited locations.", "You just ran 2 miles!" ask: "Would you like to go to Kresge first?", "Who do you think killed the Butler?" request: "Please run a little faster". greet: "Hello", "Goodbye" command: "Go to Kresge." Example 2: Others seen in Personage's folders assert: "SELECTION_X2 is a MOD_X restaurant." question: "Did you say SELECTION?" reject: "I did not find any restaurant matching your request." request: "Can you try again?", "What kind of X are you looking for?" confirm: "Let me confirm" bye: "Thank you for using this system. Before you hang up, could you rate it by giving a number between 1 and 10 out of 10?" hello: "How may I help you?" inform: "SELECTION_X1 is a great X." select: "Would you like a venue with X_1 or X_2 star?" suggest: "For example, a X1."