LLM PROMPT PILOT v0.2

---

ABOUT PROMPT PARAMETERS

Large language models only generate responses based on the statistical relationships between words and phrases in their training data. That said, asking for responses in the right way can help a model better simulate comprehension what a user wants in a reply.

Customizing prompt parameters involves tailoring the input or instructions provided to a large language model (LLM) - e.g. OpenAI's ChatGPT, Google Gemini, Anthropic's Claude - to elicit more accurate and relevant responses. By refining the prompts, users can guide the model to produce desired outputs. Here's a summary of how customizing prompt parameters contributes to better responses:

Clarity of Instruction: Clearly specifying the desired format or type of response in the prompt helps the model understand the user's expectations, reducing ambiguity and generating more relevant answers.

Contextual Information: Including relevant context in the prompt can help the model understand the background of the query. This ensures that the responses take into account the specific context provided, leading to more coherent and accurate answers.

Iteration and Experimentation: Users can refine their prompts through an iterative process, experimenting with different instructions to observe how the model responds. Repeating these steps allows for fine-tuning and optimization of the input to achieve better outcomes.

Task-specific Guidance: When using language models for specific tasks, providing task-specific instructions in the prompt helps the model focus on the intended goal.

Balancing Creativity and Specificity: Users can guide the model to be creative while ensuring that the generated content remains relevant and aligned with their requirements.

Handling Sensitivity: Customizing prompts helps in addressing sensitivity concerns by providing explicit instructions on the desired tone, approach, or avoidance of certain topics. This is particularly useful in generating content that is appropriate for ethical or contextual considerations.

ABOUT MANUALLY ASKING FOR TECH PARAMETERS

As application programming interfaces (APIs) into LLMs evolve, developers will have more direct access to tweak responses outside of the natural language text within the prompt itself. Version 0.2 of this app does not allow use of these parameters, but options for their use have been retained as placeholders once a future API-driven version is functional. See the end of this section for intended use.

For more information about integrating with LLM APIs, see:

• platform.openai.com/docs/api-reference
• labs.google
• support.anthropic.com/en/collections/4078533-api-prompt-design

Temperature: This setting controls the randomness of an LLM model's output. Higher values make the output more creative and random, while lower values make it more predictable and deterministic. If your responses do not seem to make any sense, lower the temperature setting.

Diversity Penalty: Tokens are the individual units that make up a piece of text, and they help computers understand and process language. (Think of tokens here as the number of words in a sentence, although the model might not count some duplicate or simple words.) Under this parameter, tokens that have already been used in the response are penalized, making the final output more varied. Setting higher values leads to more diverse text, while lower values may use more repetitive phrases.

Max Tokens: Limits the length of the response generated by the model to the specified number of tokens. As a rough rule of thumb, one token equate to about 4 characters or 3/4 words, so 100 tokens estimates to about 75 words, or a typical paragraph. (More info »)

Hyperparameters: Some APIs allow for directly setting the number of times the machines "think" about a response before outputting a final reply. In the training phase (which occurs before you use the model for generating responses), the model goes through multiple epochs, where it sees the entire training dataset. More epochs during training can allow the model to learn more complex patterns from the data, potentially improving its ability to generate contextually relevant responses. Fine-tuning is the process of training a pre-existing model on a more specific dataset. Increasing fine-tuning epochs allows the model to adapt better to the specifics of the new data, potentially making it more suitable for certain tasks or contexts.

Remove Stop Words: Borrowing a trick from the Natural Language Processing Toolkit (NLTK), this setting removes some English-language stopwords, words that are really common in a language but usually don't carry much meaning on their own. Think of the glue that holds sentences together: "the," "and," "is," and "in." When working with language, people often remove stopwords to focus on the more important and meaningful words in a piece of text. In this case, the option is given to reduce the token/character count passed to an LLM as a way to get around input limits. (It's a cheap hack; its utility may be marginal.)

Use Cases on API Hyperparameters: Whenever the API-based version of this app is live, the following templates can be coded into the tool for direct reference to each language model. (Credit due to Anca Budau for these templates.)

Blog Post
Temperature: 0.5 to 0.7 for a balance between creativity and coherence.
Diversity_penalty: 1 to ensure diverse output without compromising readability.
Max tokens: Set according to your desired word count or limit to keep responses concise.

Newsletter Email
Temperature: 0.3 to 0.5 for focused and coherent outputs.
Diversity_penalty: 0.5 to 1 to avoid repetition while maintaining readability.
Max tokens: Set according to the desired length of your email content.

Social Media Caption
Use these settings: Temperature: 0.7 to 1 for more creative and diverse outputs.
Diversity_penalty: 1 to 2 for maximum diversity without sacrificing too much coherence.
Max tokens: Set a limit that corresponds to the platform's character limit.

UX writing
Temperature: 0.2 to 0.4 for highly focused and coherent outputs.
Diversity_penalty: 0 to 0.5 to maintain clarity and avoid repetitive phrases.
Max tokens: Set a strict limit to keep instructions and labels concise.

Creative storytelling
Temperature: 0.7 to 1 for maximum creativity and variety.
Diversity_penalty: 1 to 2 for a wide range of unique phrases and ideas.
Max tokens: Set according to your desired story length or scene.

Formal business writing
Temperature: 0.2 to 0.4 for focused, coherent, and conservative outputs.
Diversity_penalty: 0 to 1 for balanced diversity without affecting readability.
Max tokens: Set a limit based on the desired length of your business document.

ABOUT THIS PROJECT

The goal here is to illustrate how users may refine large language model prompts and responses, tailoring them more precisely to specific requests related to different target audiences.

The initial proposal for this simple web app was to show how one of the roles of a media producer could be almost fully automated through the use of large language models, specifically OpenAI's Chat GPT.

The scope of this tool was designed for the CUNY Graduate Center's Large Language Models and ChatGPT course by Michelle A. McSweeney - specifically, the Fall 2023 semester's OpenAI Playground lab and the Oct. 30 class discussion on Labor & Creative Production led by Kelly Cunningham, Ruby Chu, and Michael Smith (notes, presentation). The in-class exercise highlighted the vulnerability of several jobs from my past career, notably News Analysts, Reporters, and Journalists (27-3023.00).

For example, at a major network news organization in the mid-2010s, it was typical to employ producers who would watch news wires, broadcasts, and Twitter feeds in order to quickly and accurately produce the following:

1. text-based articles for publishing to one or more news-branded or lifestyle websites,
2. review and optimize articles for best SEO performance,
3. create a series of social media posts for promotion of said web articles via Facebook, Twitter, and LinkedIn,
4. create a television broadcast script for quick production of a web-exclusive video segment,
5. forward a summary recap to editors at various syndication partners and topical-interest sites, and,
6. log a note off the published posts in a daily rundown.

Depending on the story, these steps might take a producer from 10 minutes to an hour to edit, or longer for actively developing news. Now with such public LLM-based tools OpenAI's Chat GPT, Google Gemini, Anthropic Claude, and many others, such tasks may be done in seconds.

While automating receptive tasks, increasing efficiency, and helping fine-tune the performance of large language models may be all fine and good, such a disruptive change in is having an immediate impact on a whole class of creative professionals. For example, take the rather clickbait-savvy headline "I Lost My Job to ChatGPT and Was Made Obsolete. I Was Out of Work for 3 Months Before Taking a New Job Passing Out Samples at Grocery Stores." by Emily Hanley, posted July 19, 2023, on Entrepreneur.com.

Credits

Many thank Bianca C. Calabresi and Ismerlyn Gonzalez who provided feedback, including Ismerlyn's highlight on the risk "it could further help some nefarious actors spread disinformation/misinformation" - one of the points to illustrate here with such technology.

Curated Readings (from Dr. Michelle McSweeney's Syllabus)

• Eloundou, et al., 2023. OpenAI GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
• May 7, 2023, The Economist. Your job is (probably) safe from artificial intelligence
• Lohr, April 10, 2023. NYTimes A.I. Is Coming for Lawyers, Again
• Klinova & Korinek, Aug 17, 2023. Brookings. Unleashing possibilities, ignoring risks: Why we need tools to manage AI's impact on jobs
• O'Reilly & Zahidi, Sept 2023. World Economic Forum. Jobs of Tomorrow: Large Language Models and Jobs

---

MATTHEW STANTON
pingstanton@gmail.com
pingstanton.com/work
github.com/pingstanton/