5 to 8 minute read
Playbooks by Program Type
Part 1: Awards
01
Planning an Awards Program
The many names these programs go by, and the single shape they almost all share.
5–8 min read
If you're reading this, chances are someone in your organization has recently said some version of "we should really recognize the best of what we do." That instinct, wanting to spot, celebrate, or fund the best work in a field, is the starting point for every award, competition, grant, or evaluation program that has ever existed. Before we talk about software, timelines, or judging panels, it is worth pausing on what these programs actually are, because the label changes far more than the substance.
In plain terms: An awards program recognizes excellence rather than distributing funding, so planning centers on categories, judging criteria, and how winners get announced, not on financial due diligence or eligibility documentation the way a grant program requires.
- Start with what you’re actually recognizing. Before naming categories, write down, in a sentence, what “winning” is supposed to mean for your program. A “Most Innovative” category and a “Best Overall” category aren’t different labels for the same judgment, they ask judges to weigh completely different things. If you can’t articulate the distinction in a sentence, your judges won’t be able to either, and you’ll get inconsistent scoring as a result.
- Decide one-time versus annual early, it changes the whole structure. A one-time program can build a bespoke process. An annual program benefits from locking a consistent structure entrants come to recognize and trust year over year, changing your category names or judging criteria every year erodes the credibility you’re trying to build. If you’re not sure yet whether this becomes annual, design it as if it will, it’s easier to simplify a repeatable structure than to retrofit consistency later.
- Budget real time for the announcement, not just the judging. Organizers consistently underestimate this. A winners’ ceremony, a press release, a social media rollout, a printed certificate, these take longer to plan than the judging round itself in most programs, and they’re usually the part entrants and sponsors actually remember. If you’re planning any kind of public announcement, start that planning in parallel with judging, not after it.
- Decide your sponsor structure before you open entries, if you’re using one. Sponsored categories change what you can promise entrants (visibility, specific prizes) and what you owe sponsors (logo placement, category naming rights). Retrofitting a sponsor into an already-designed category structure is a common source of late scrambling.
Recap
- Write down what each category is actually judging before naming it, vague categories produce inconsistent scoring.
- Decide one-time vs. annual early, and design for consistency if there’s any chance it becomes annual.
- The announcement and ceremony often take more planning time than judging itself. Start that work in parallel, not after.
- Lock your sponsor structure before entries open, not after.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1. What is the most common planning mistake described in this section?
2. Why does an annual program benefit from a consistent structure?
3. It is fine to add a sponsored category after the rest of the category structure is already designed.
02
Judging Criteria for Awards
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Awards criteria usually balance objective achievement with subjective quality, which makes a clear, explicitly weighted rubric more important here than in program types with more purely numeric criteria.
- Objective and subjective criteria need to be named separately. “Revenue growth” is objective, you can point to a number. “Creativity of approach” is subjective, two reasonable judges can disagree. Both are legitimate criteria, but mixing them into one vague category like “impact” makes it impossible for a judge to know whether they’re scoring the number or their gut reaction to it. Separate them into distinct rubric lines.
- Weight your criteria on purpose, not by default. The common failure mode is listing five criteria and letting them implicitly count equally, without ever deciding whether that’s actually right. If originality matters twice as much as polish to your program, say so numerically. Two judges without a shared, explicit weighting will produce inconsistent results even when they’re using the exact same rubric and the exact same five words.
- Decide how much weight, if any, past recognition carries. A common and controversial question in awards judging is whether a previous winner, or an entrant who’s well known in the field, should have an edge. Most well-run programs explicitly instruct judges to score the current entry only, not the entrant’s reputation, but this needs to be stated, not assumed, because reputation bias creeps in quietly otherwise.
- Build in a mechanism for judges to flag when they lack the expertise to score something fairly. Awards programs often span a wide range of sub-fields under one category. A judge scoring a “Best Product” category across five different industries may have real expertise in only one or two. Letting a judge flag low-confidence scores, or abstain from ones truly outside their expertise, produces more honest results than forcing a score anyway.
Recap
- Separate objective and subjective criteria into distinct rubric lines rather than blending them into vague categories.
- Weight criteria explicitly and on purpose, not by unstated default equal weighting.
- Decide, and state clearly to judges, whether an entrant’s past recognition or reputation should factor in, most programs say it shouldn’t.
- Give judges a way to flag or abstain from entries outside their real expertise.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1. Why should objective and subjective criteria be listed as separate rubric lines?
2. What is the common failure mode in criteria weighting?
3. Most well-run awards programs instruct judges to factor in an entrant past recognition.
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- What happens if two entries receive the exact same score? “Decide your tie-break rule before you need it, not in the moment. Common approaches: fall back to a specific secondary criterion (whichever entry scored higher on “impact,” for instance), give the lead judge or committee a deciding vote, or name co-winners if your program structure allows it. Whatever you choose, document it in your judging guidelines so it doesn’t look like an arbitrary call after the fact.
- Can an organizer’s own team or company also enter? This is best avoided outright. If it’s genuinely unavoidable, the minimum bar is full disclosure to entrants that an organizer-affiliated entry exists, combined with that organizer being completely recused from any judging, scoring, or deliberation involving their own entry. Even with those safeguards, expect some entrants to view the program’s credibility skeptically, that’s a real cost worth weighing before allowing it.
- Can the same organization or individual win multiple years in a row? Nothing inherently prevents this if the entries genuinely earn it each time, judging should reward merit, not enforce artificial variety. That said, some programs deliberately add a rule limiting consecutive wins in the same category, partly to keep the program feeling fresh and accessible to new entrants, and partly to avoid the appearance of favoritism toward repeat winners
- What if a category attracts far fewer entries than expected? Decide in advance whether a category needs a minimum entry count to run at all. Awarding a “winner” from a pool of two or three weak entries can undermine the credibility of the category, and by extension the program. Better to fold a thin category into a broader one, or skip awarding it that cycle, than to hand out a hollow win.
Recap
- Set your tie-break method in writing before judging starts.
- Organizer entries need full disclosure and complete recusal at minimum, and still carry a credibility cost worth weighing.
- Repeat winners are fine on merit; some programs still cap consecutive wins by design choice.
- Decide a minimum viable entry count per category in advance, rather than awarding a hollow win from a thin pool.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What should be documented in judging guidelines regarding ties?
2.What is the minimum safeguard if an organizers own entry is unavoidable?
3. A category should always award a winner even if only two weak entries were submitted
Part 2: Competitions
01
Planning a Competition
The many names these programs go by, and the single shape they almost all share.
5–8 min read
In plain terms: Competitions often involve a live or time-boxed element, matches, performances, demonstrations, so planning needs to account for both the submitted material and the event itself, two very different things to schedule around.
- Decide upfront whether judging happens on submitted material alone, or also on a live component. This single decision shapes almost everything else: your timeline, your venue needs (if any), your judge briefing, and your contingency planning. A competition that’s purely submission-based can be judged asynchronously, on each judge’s own schedule. One with a live component cannot, everyone, judges included, needs to be in the same place, physically or virtually, at the same time.
- If there’s a live element, build extra buffer into your schedule. Live events run behind schedule far more often than digital-only judging does, a delayed start, a technical hiccup, a longer-than-expected Q&A, these compound quickly. Pad your schedule more generously than feels necessary, and communicate that buffer to judges and competitors so nobody’s caught off guard by a shifted timeline.
- Plan your venue or platform requirements early, not as an afterthought. Whether it’s a physical stage or a video conferencing setup, the technical requirements (audio, lighting, screen sharing, recording) need testing well before competition day, not discovered live. A quick technical rehearsal with a subset of judges and competitors catches most problems cheaply.
Recap
- Decide submission-only vs. live-component judging early, it shapes your entire timeline and logistics.
- Live elements need more schedule buffer than pure digital judging, pad generously and communicate it.
- Test venue or platform technical requirements before competition day, not during it.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What tends to run behind schedule more often than digital-only judging?
2. It is fine to decide the judging format (submission-only vs. live) after entries arrive.
3.What is recommended before competition day for live or virtual events?
02
Judging Criteria for Awards
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Competition criteria often mix a measurable component (a score, a time, a result) with a qualitative one (execution, presentation), and both need clear, separate weighting so neither quietly dominates the other.
The same shape, many names
Despite the different names, most of these programs share the same underlying shape: someone submits something, someone else reviews it against criteria, and a decision gets made. Once you see that shape, the different labels start to look like variations on a theme rather than separate categories:
- Separate the measurable from the qualitative in your rubric, don’t let one silently absorb the other. A race has a time. A pitch competition has content and delivery. If your rubric blends “did they hit the technical requirement” with “how well did they present it” into a single number, you lose the ability to tell whether a low score came from missing the mark technically or just presenting poorly, information you need if you ever want to give useful feedback.
- Decide whether judges score in real time or review recordings afterward. Real-time scoring is faster and cheaper to run, but leaves judges with no ability to reconsider or compare a strong early performance against a stronger one that comes later in the day. Recorded review is slower and requires more infrastructure, but produces more consistent, comparable scoring because judges can review multiple entries side by side before finalizing anything.
- If judges are scoring live, give them a structured way to take notes in the moment. Memory fades fast across a full day of back-to-back performances or demos. A simple structured note field tied to each rubric criterion, filled in immediately after each entry, produces far more defensible scores than judges trying to reconstruct their reasoning at the end of the day.
Recap
- Criteria often mix measurable results with qualitative assessment, keep them as separate rubric lines.
- Real-time scoring is faster but leaves less room to reconsider; recorded review is slower but more consistent.
- Give live judges a structured way to take notes immediately after each entry, not at the end of the day.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1. What is a tradeoff of real-time scoring versus recorded review?
2. Competition judging never involves a measurable component.
3.Why should judges take structured notes immediately after each live entry?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
In plain terms: Competition criteria often mix a measurable component (a score, a time, a result) with a qualitative one (execution, presentation), and both need clear, separate weighting so neither quietly dominates the other.
- Can an organizer’s own team also compete? Generally avoided, for the same conflict-of-interest reason as awards programs, and it’s harder to manage cleanly in a live setting, where recusal from “just one entry” is more visible and awkward than it is in an asynchronous, submission-based judging process.
- What if a competitor is late to a live component? Set a grace period in advance, five minutes, ten minutes, whatever fits your format, and apply it consistently to everyone. Exceptions made in the moment, even well-intentioned ones, tend to look unfair after the fact, especially to competitors who arrived on time.
- What happens if a competitor’s equipment or technology fails during a live component? Decide your policy before the event: a brief grace period to resolve it, a backup demo or recording, or moving them to the end of the queue. Whatever you choose, state it in the competition rules upfront so it’s not a judgment call made under pressure in the moment.
Recap
- Organizer participation raises the same conflict issue as awards, and it’s harder to manage in a live setting.
- Grace periods for lateness should be fixed rules, decided in advance, not judgment calls made live.
- Have a stated policy for technical or equipment failure during a live component, decided before the event.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.When should a grace period for lateness be decided?
2.Exceptions made in the moment for lateness tend to look fair to everyone.
3. What should happen with a technology-failure policy?
Part 3: Grants
01
Planning a Grant Program
Planning a Grant Program
5–8 min read
In plain terms: Grant programs decide who receives funding, so the review process usually needs more rigor and confidentiality than an awards program, since the outcome has real financial consequences for applicants and real accountability requirements for whoever's distributing the money.
- Define your funding criteria and eligibility rules with real precision. Vague criteria (“supports community impact”) invite a flood of unsuitable applications that waste reviewer time and frustrate applicants who genuinely didn’t fit but couldn’t tell from the description. Specific, concrete eligibility rules (geographic scope, organization type, funding range, project stage) do far more filtering work upfront than any amount of careful review can do after the fact.
- Decide your review structure, single-stage or multi-stage, before applications open. A single-stage review works for smaller funding pools with fewer applicants. A multi-stage process (initial screening, then a deeper review of finalists) works better at scale but needs to be designed and communicated from the start, adding a stage midway through is confusing and can feel unfair to applicants who already invested time under the original process.
- Build in a documented conflict-of-interest process from day one. Grant reviewers are often part of the same professional community as applicants, that’s often exactly why they’re qualified to review. Have reviewers declare potential conflicts before seeing applications, not after, and have a clear recusal process ready to use.
Recap
- Grant review needs more rigor and confidentiality than most awards programs, given the real financial stakes.
- Precise eligibility criteria reduce unsuitable applications more effectively than review-stage filtering.
- Lock the review structure (single-stage vs. multi-stage) before applications open.
- Build a documented conflict-of-interest process in from the start, given how connected reviewers and applicants often are.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Why does grant review typically need more rigor than an awards program?
2. Vague eligibility criteria tend to reduce the number of unsuitable applications.
3.What should be decided before applications open?
02
Judging Criteria for Grants
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Grant criteria typically weigh mission fit, feasibility, and need, alongside the applicant's actual capacity to deliver on what they're proposing, four distinct questions that are easy to blur together into one vague "is this a good application" impression.
- Feasibility deserves particular, deliberate attention. A compelling idea from an applicant who can’t realistically execute it is one of the most common traps in grant review, reviewers get excited about the vision and under-scrutinize whether the applicant actually has the track record, team, or resources to pull it off. Build a specific rubric line for feasibility and capacity, separate from mission fit, so it can’t get glossed over.
- Decide whether financial need is its own weighted criterion or a threshold requirement. Some programs treat need as pass/fail (you either qualify or you don’t) and then evaluate remaining applicants purely on merit. Others weight need as an ongoing factor throughout scoring, favoring the most under-resourced qualified applicant. These produce meaningfully different outcomes, decide deliberately which model fits your program’s actual goals.
- Give reviewers a way to distinguish “doesn’t fit our mission” from “fits our mission but is a weaker proposal.” These get the same low score if your rubric doesn’t separate them, but they mean very different things for the applicant and for how you might advise them going forward.
Recap
- Criteria usually include mission fit, feasibility, need, and applicant capacity, as distinct rubric lines.
- Feasibility and capacity to execute deserve as much deliberate scrutiny as the idea itself.
- Decide whether financial need is a pass/fail threshold or an ongoing weighted factor, the two produce different outcomes.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1. What common trap does this section highlight in grant review?
2. Financial need is sometimes treated as a pass/fail threshold rather than an ongoing weighted factor.
3.Why separate "doesn't fit our mission" from "fits but is weaker" in scoring?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- Can a grant applicant reapply after being rejected once? Usually yes, unless the program explicitly states a waiting period. Rejection in a competitive grant round often reflects the strength of that particular applicant pool as much as anything else, it isn’t typically treated as a permanent judgment.
- Can one organization receive multiple grants from the same program? Depends entirely on the program’s rules. Some programs cap funding per organization per cycle to spread resources across more recipients, others allow multiple concurrent grants if each one is independently justified. Decide and state this explicitly rather than leaving it ambiguous.
- What if an applicant’s circumstances change significantly between applying and a funding decision? Build in a mechanism for applicants to flag material changes (a leadership change, a shift in project scope) before a final decision is made. Making a funding decision based on information that’s gone stale creates real problems, for the funder and the recipient, down the line.
Recap
- Reapplication after rejection is usually allowed, rejection isn’t typically a permanent bar.
- Multiple grants to one organization depends entirely on program-specific rules, decide and state this explicitly.
- Give applicants a way to flag material changes before a final funding decision is made.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Is rejection from a grant program usually a permanent bar to reapplying?
2.Every grant program handles multiple awards to one organization the same way.
3.What should applicants be able to do if their circumstances change materially?
Part 4: Scholarships
01
Planning a Scholarship Program
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Scholarship programs evaluate people, not just projects, so eligibility verification (enrollment status, academic standing, program of study) plays a bigger role here than in most other program types, and the timeline has to bend around someone else's academic calendar, not your own
- Decide what proof of eligibility you’ll require, and when, before applications open. Transcripts, enrollment verification letters, proof of program acceptance, decide exactly what’s needed and build the request into the application itself. Chasing down documentation after the fact, once you’re trying to finalize a decision, slows everything down and puts pressure on both your team and the applicant at the worst possible time.
- Build your timeline around the applicant’s calendar, not just your own internal schedule. Application deadlines, decision dates, and enrollment deadlines all need to line up with the academic calendar of the institutions your applicants attend, tuition deadlines and enrollment confirmation dates aren’t flexible on the institution’s end, so your scholarship timeline has to work backward from those fixed points.
- Decide how you’ll verify identity and prevent application fraud, especially for larger or more valuable scholarships. As scholarship value or prestige increases, so does the incentive for misrepresentation. Basic verification steps (institutional confirmation, reference checks) matter more the higher the stakes.
Recap
- Eligibility verification plays a larger role in scholarships than in most program types.
- Decide required documentation upfront and build it into the application, don’t chase it down later.
- Timelines should follow the applicant’s academic calendar, not just your own internal schedule.
- Consider identity and fraud verification steps proportional to the scholarship’s value.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What plays a bigger role in scholarships than in most other program types?
2. It is fine to decide what documentation you need after applications have already closed.
3.What should scholarship timelines be built around?
02
Judging Criteria for Scholarships
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Scholarship criteria usually weigh academic merit, financial need, or both, and the right balance depends entirely on what the scholarship is meant to achieve, a distinction that needs to be decided deliberately, not left implicit.
- Merit-based and need-based scholarships require genuinely different evaluation processes, treating them the same undermines the goal. A merit scholarship evaluated with heavy weight on financial hardship stops rewarding merit. A need-based scholarship evaluated primarily on GPA stops reaching the applicants it was designed for. Get clear on which one (or what blend) your scholarship actually is before writing the rubric, not after.
- Essays and personal statements often carry real weight, so a shared standard for evaluating writing matters as much as numeric criteria. Without a shared standard, essay scoring becomes the most inconsistent part of the whole process, one reviewer rewards polished writing, another rewards a compelling personal story, and applicants get wildly different scores depending purely on who reads their essay. A rubric that separates content (does it address what was asked) from writing quality helps reviewers score more consistently.
- If you’re using financial documentation to assess need, decide your verification standard in advance. Self-reported income, tax documentation, financial aid office confirmation, these carry different burdens for applicants and different levels of certainty for you. Choose deliberately based on your scholarship’s stakes and your program’s capacity to process documentation.
Recap
- Criteria typically weigh academic merit and financial need, in some deliberate balance decided upfront.
- Merit-based and need-based scholarships need different evaluation approaches, don’t default to treating them the same.
- Essays and personal statements need a shared scoring standard, separating content from writing quality.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What happens when merit-based and need-based scholarships are evaluated the same way?
2. Essays and personal statements are sometimes weighted as heavily as numeric criteria.
3.What helps reviewers score essays more consistently?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- What if a scholarship winner doesn’t meet the enrollment requirement afterward? Most well-run programs define a fallback in advance, commonly rescinding the award and moving to the next-ranked applicant. Decide this before you ever announce a winner, not after you’re facing the actual situation with a real person involved.
- Can a rejected applicant apply again next cycle? Usually yes, unless the program states a waiting period. A rejection in one cycle often reflects the strength of that specific applicant pool, not a permanent judgment on the applicant.
- What if two family members from the same household both apply? Decide in advance whether this is allowed, and if so, whether there’s a cap on total awards per household. This comes up more often than organizers expect, especially for scholarships tied to a specific school, community, or employer.
Recap
- Define your enrollment-requirement fallback before it’s needed, not after a real case comes up.
- Reapplying next cycle is usually allowed.
- Decide your household/family-member policy in advance, this comes up more often than expected.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.When should a fallback for a winner not meeting enrollment requirements be decided?
2.Most scholarship programs permanently ban an applicant after one rejection
3.What should be decided regarding multiple applicants from the same household?
Part 5: Abstracts
01
Planning an Abstract Program
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Abstract programs collect short summaries for review before a conference presentation or publication, so the timeline needs to work backward from a fixed external event date that you almost certainly don't control.
- Set your abstract deadline with real buffer before the conference or publication date. Reviewers need time to read and score submissions, authors of accepted abstracts often need time to prepare a full paper or presentation, and conference organizers are rarely flexible about moving their own printed program or schedule. Work backward from the fixed date and build in more buffer than feels necessary, review always takes longer than planned.
- Decide upfront whether you’re using single-blind, double-blind, or open peer review. This affects what information authors need to omit from their submission (their name and institution, for double-blind) and what infrastructure you need to manage that anonymization. Deciding this after abstracts start arriving means retrofitting anonymization onto submissions that already include identifying information, a much harder problem.
- Recruit reviewers with real subject-matter breadth, not just volume. Abstract programs often span multiple sub-topics under one call for submissions. A reviewer pool that’s deep in one area but thin in others produces inconsistent review quality across topics. Map your expected submission topics against your reviewer expertise before opening the call, not after submissions reveal the gap.
Recap
- Abstract timelines work backward from a fixed external event date, usually one you don’t control.
- Build in more review buffer than feels necessary, review consistently takes longer than planned.
- Decide your peer review type (single-blind, double-blind, open) before submissions open, retrofitting anonymization later is much harder.
- Map reviewer expertise against expected submission topics before opening the call.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What does an abstract programs timeline typically work backward from?
2. Conference organizers are usually flexible about moving their dates to suit abstract review
3.Why decide peer review type before submissions open?
02
Judging Criteria for Abstracts
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Abstract review typically weighs originality, methodological soundness, and relevance to the conference or publication's scope, more than presentation polish, since an abstract is a compressed summary, not a finished piece of writing.
- Reviewers need real subject-matter expertise to judge soundness accurately. Because abstracts are short, there’s little room for a reviewer to infer rigor from context the way they might with a full paper. General reviewers without domain knowledge tend to default to evaluating clarity of writing instead of the substance underneath, which produces scores that reward polished writing over sound methodology, exactly backward from what most programs actually want.
- Relevance to scope matters more here than in most program types. A technically strong, well-written abstract on the wrong topic still gets rejected, and this needs to be communicated clearly to reviewers as a distinct criterion, not folded into a general quality score. Otherwise reviewers sometimes score a strong but off-topic abstract highly simply because it’s well done, creating downstream confusion when it’s rejected anyway.
- Decide how much weight, if any, to give to whether the proposed work is already completed versus still in progress. Some conferences want to feature finished research, others specifically want to showcase early-stage or in-progress work. Neither is wrong, but reviewers need to know which one your program wants so they’re not penalizing in-progress work that was never meant to be finished yet.
Recap
- Criteria center on originality, methodological soundness, and relevance to scope, more than writing polish.
- Reviewers typically need real subject-matter expertise, general reviewers tend to reward clarity over substance.
- Relevance to scope should be a distinct criterion, a strong but off-topic abstract still gets rejected.
- Clarify whether completed or in-progress work is preferred, so reviewers don’t penalize the wrong thing.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What do abstract reviewers typically need that general judges may lack?
2. A technically strong abstract on the wrong topic is usually still accepted.
3.Why should completed vs. in-progress work status be clarified for reviewers?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
In plain terms: Abstract review typically weighs originality, methodological soundness, and relevance to the conference or publication's scope, more than presentation polish, since an abstract is a compressed summary, not a finished piece of writing.
- What is the difference between an abstract and a full paper submission? An abstract is a short summary, typically a few hundred words, reviewed first to determine whether the work fits the program and merits inclusion. A full paper, where required, is the complete work, sometimes requested only after the abstract has already been accepted, so authors aren’t asked to write a full paper before knowing whether there’s any chance of acceptance.
- Can an abstract be revised after submission? Depends entirely on the program. Some allow revisions up until the submission deadline itself, then treat the abstract as final. Others allow a brief revision window after initial reviewer feedback, before a final decision. State your policy clearly in the call for submissions so authors aren’t guessing.
- What happens if an accepted abstract’s author can no longer attend the conference? Decide your policy in advance, some programs allow a co-author to present instead, others require the original author to present or the acceptance is withdrawn. This comes up more often than organizers expect, and having a stated policy avoids an awkward case-by-case negotiation.
Recap
- An abstract is a short summary, reviewed before any full paper is requested.
- Revision policy after submission varies by program, state it clearly in the call for submissions.
- Decide your policy on presenter substitution in advance, this comes up regularly.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What is an abstract, relative to a full paper?
2.Every abstract program allows revisions after submission.
3.What should be decided in advance regarding presenter substitution?
Part 6: Hackathons
01
Planning a Hackathon
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Hackathons are time-boxed team competitions, so planning centers on the event window itself, team formation, tracks or themes, and a compressed judging turnaround right after submissions close, everything happens fast and close together.
- Decide your tracks or themes early and communicate them clearly. Teams often form specifically around a track, sometimes before the event even starts, so late changes to available tracks can strand teams mid-formation or mid-project. If you’re not certain about your final track list, communicate a provisional list early and confirm it well before the event, not the morning it starts.
- Plan judging to happen fast, and staff for that speed. Hackathon submissions are usually judged within hours or a day of the event ending, not weeks later, since momentum and relevance fade quickly. This means recruiting enough judges to review a potentially large volume of projects in a very short window, and giving them a streamlined process (short demos, structured scoring) rather than the deep, deliberate review a grant or abstract program might use.
- Decide your team formation rules before the event starts. Can people register solo and be matched into a team? Is there a minimum or maximum team size? Can someone bring pre-written code from before the event? These questions need answers before participants show up expecting a certain format, changing the rules on event day creates confusion and a sense of unfairness.
Recap
- Hackathon planning centers on the event window and team formation.
- Announce tracks or themes early and confirm them well before the event, teams build around them.
- Judging turnaround is typically fast, hours to a day, staff and structure your process for that speed.
- Decide team formation rules (solo registration, team size limits, pre-written code policy) before the event starts.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Why should hackathon tracks or themes be announced early?
2.Hackathon judging typically takes several weeks after the event ends.
3.What should be decided before the event regarding teams?
02
Judging Criteria for Hackathons
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Hackathon criteria typically weigh technical execution, creativity, and how well the project addresses its chosen track, tempered by the reality that it was built in a very short window, a standard that would be unreasonable for any other program type in this book.
- Judges need explicit context on the time constraint, built into the rubric itself. A rougher demo built in 24 hours should be evaluated differently than a polished product built over months, but judges pulled in from industry sometimes default to their normal professional standards unless explicitly told otherwise. State clearly in judge briefing materials that execution should be scored relative to the time constraint, not against a general industry bar.
- Decide whether judges score the live demo itself or the underlying project, and make sure judges know which. Live demos are common and exciting, but they’re also where things go wrong, a bug appears, a server hiccups, a feature that worked in testing fails on stage. If judges are meant to look past a demo glitch and evaluate the underlying work, they need to be told that explicitly, otherwise a strong project can be unfairly punished for a five-second technical hiccup.
- Build a structured, fast scoring process, since judges typically move between many short demos in quick succession. A simple checklist-style rubric that judges can complete in the two or three minutes right after each demo produces far more accurate scoring than judges trying to remember and compare fifteen demos at the end of a long day.
Recap
- Criteria typically cover technical execution, creativity, and track fit, evaluated against the short build window, not general industry standards.
- State explicitly in judge briefings that scoring should account for the time constraint.
- Decide and communicate whether judges score the live demo or the underlying project, demos are prone to glitches.
- Use a fast, structured scoring process judges can complete immediately after each demo.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What should judges account for when scoring hackathon projects?
2. A live demo going wrong should always be scored the same as the underlying project working perfectly.
3.Why use a fast, structured scoring process for hackathon judging?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- Can a hackathon team change members mid-event? Depends on the program’s rules, some allow it early on, in the first hour or two, while teams are still forming, but most restrict changes once the event is properly underway, since a mid-event swap can raise fairness questions about who actually contributed to the final project.
- What if a team’s project doesn’t work during the live demo? Most well-run programs allow judges to consider a recorded backup demo or video captured earlier, precisely because live technical failure is common and not always a reflection of the work itself. Decide and communicate this backup-demo policy in advance, and remind teams to prepare a backup recording as a standard part of their submission checklist.
- Can a team use code or components they wrote before the event started? This varies by program and needs a clear, stated policy, some hackathons require everything to be built from scratch during the event window, others allow pre-existing components as long as the core project work happens during the event. State this explicitly in the rules, ambiguity here is a common source of disputes.
Recap
- Mid-event team changes depend on program-specific rules, usually restricted once the event is properly underway.
- Have a backup-demo policy decided before the event, live technical failure is common and shouldn’t automatically tank a score.
- State your pre-written code policy explicitly, ambiguity here is a common source of disputes
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.When should a backup-demo policy for technical failures be decided?
2.Most hackathon programs freely allow team changes at any point during the event.
3.What is a common source of disputes in hackathons?
Part 7: Talent
01
Planning a Talent Program
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Talent programs evaluate people directly, often through video or live audition, so technical submission requirements and a clear audition or review process matter more here than in project-based programs, where the format is usually more standardized.
- Specify video and file requirements precisely, format, length, resolution, and file size. Talent submissions are especially prone to technical rejection if requirements are vague, an applicant who spends real effort on a performance can lose out entirely because their file wouldn’t upload or play correctly. Be explicit, and provide simple guidance (recommended recording apps or settings) for applicants who may not be technically sophisticated.
- Decide early whether review happens on submitted material alone or includes a live audition stage. This single decision affects your entire timeline and cost structure. A submission-only process can scale to far more applicants at a lower cost per applicant. A live audition stage produces richer information (how someone handles real-time direction or pressure) but is expensive and slow to run at volume, so it’s usually reserved for a smaller pool of finalists who’ve already passed an initial submission-based round.
- Think through accessibility and equity in submission requirements from the start. Requiring an expensive video setup or a professional recording space can unintentionally exclude talented applicants who don’t have access to that equipment. Where possible, make clear that content and skill matter more than production quality, and say so explicitly in your submission guidelines.
Recap
- Talent programs evaluate people, often via video or live audition, more than project-based programs.
- Precise technical requirements reduce avoidable technical rejections, provide simple guidance for less technical applicants.
- Decide submission-only vs. live audition early, it affects your entire timeline and cost structure.
- Consider accessibility in submission requirements, expensive equipment shouldn’t be an implicit barrier.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Why are precise video and file requirements especially important for talent programs?
2.Whether to include a live audition stage is a decision that can be made late without much impact.
3.What should submission guidelines make clear regarding production quality?
02
Judging Criteria for Talent
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Talent criteria typically weigh skill, presence, and originality, and because the subject is a person rather than a project, feedback tends to carry more emotional weight and deserves extra care in both scoring and delivery.
- Skill and polish aren’t the same thing, and conflating them skews results toward whoever had the better production setup. A technically rougher performance with real originality often outscores a polished but generic one under most talent rubrics, that’s usually the intent. Make sure your rubric explicitly separates raw skill and originality from production quality, so a well-lit, professionally edited but unremarkable performance doesn’t outscore raw, unpolished talent.
- Presence is real but genuinely hard to score consistently, so give judges concrete anchors, not just a vague scale. “Stage presence” or “charisma” means different things to different judges without more specific guidance. Break it into more concrete sub-elements where possible, energy, connection with the material, confidence, so judges have something more consistent to anchor a score to.
- Because feedback here is about a person’s ability, not just their output, judges should be especially deliberate about how feedback is worded. A note like “weak technique” lands very differently for a performer than “needs more work” lands for a business proposal. Where feedback is shared with applicants, review it for tone as well as content, and favor specific, actionable notes over blunt judgments.
Recap
- Criteria typically include skill, presence, and originality, evaluated as distinct rubric lines.
- Polish and skill are not interchangeable, separate them explicitly so production quality doesn’t dominate scoring.
- Break vague qualities like “presence” into more concrete, scoreable sub-elements.
- Feedback carries more emotional weight in talent judging and deserves extra care in wording and tone.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What is true about polish versus originality in talent judging?
2. Feedback wording matters less in talent programs than in project-based ones
3.What helps judges score a vague quality like "presence" more consistently?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- What if a talent applicant submits a video in the wrong format? Most well-run programs allow a short correction window before rejecting outright, since talent applicants are often less technically sophisticated than applicants to other program types, and a strict zero-tolerance policy disproportionately excludes people based on technical literacy rather than talent. Decide this correction policy in advance, and state it clearly in submission guidelines.
- Can an applicant submit more than once? Depends on the program, some allow one resubmission before the deadline if an applicant realizes they can do better, others treat the first submission as final to keep the process simple and fair across all applicants. Either approach is reasonable, but state it clearly so applicants aren’t guessing.
- What if an applicant appears to be significantly younger or older than an age category allows? Build an age-verification step into your eligibility screening if age categories are part of your program design, rather than relying on judges to flag it after the fact during scoring, which puts judges in an awkward position and can introduce inconsistency.
Recap
- Decide your wrong-format correction policy in advance, and state it clearly to reduce technical-literacy bias.
- Resubmission policy varies by program, state it clearly either way.
- Verify age-category eligibility during screening, not by relying on judges to flag it during scoring.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.When should a wrong-format correction policy be decided?
2.Every talent program allows unlimited resubmissions.
3.Where should age-category verification happen?
Part 8: Speakers
01
Planning a Speaker Selection Program
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Speaker programs choose people to fill a limited number of slots at an event, so planning has to account for both the review process and a fixed event date that won't move, plus the very real logistics of getting a real person prepared and on a stage.
- Work backward from the event date, and build in real time between acceptance and the event itself. Speakers often need weeks or months to prepare content, arrange travel, and coordinate logistics with your event team, a acceptance notification sent just a few weeks before the event puts real strain on speakers and increases the odds of last-minute withdrawal or underprepared sessions.
- Decide your topic or track structure early, so submissions map cleanly onto the actual slots available at the event. If you have eight speaking slots across three tracks, know that before you open the call for proposals, otherwise you risk collecting forty strong proposals for one track and three for another, forcing awkward last-minute rebalancing or difficult rejections of genuinely strong content that just landed in an oversubscribed track.
- Plan for a formal agreement with accepted speakers, covering the basics: what’s expected of them, what you’re providing, and what happens if either side needs to change plans. This doesn’t need to be an elaborate contract, but a simple written confirmation of expectations (session length, AV needs, any compensation or travel support) prevents a surprising number of last-minute misunderstandings.
Recap
- Speaker programs plan backward from a fixed, non-negotiable event date.
- Leave real time between acceptance and the event, speakers need weeks or months to prepare and arrange logistics.
- Define topic or track structure early so submissions map onto real available slots.
- Confirm expectations with accepted speakers in writing, even informally, to prevent last-minute misunderstandings.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What does a speaker program typically plan backward from?
2.Speakers generally need little time between acceptance and the event itself.
3.Why define topic or track structure before opening the call for proposals?
02
Judging Criteria for Speakers
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Speaker selection typically weighs topic relevance, expertise, and presentation ability, balanced against the practical reality of a fixed number of available slots, which makes this one of the more constrained judging processes in this book.
- Since slots are limited, reviewers often have to choose between two strong candidates on the same topic. Deciding in advance how you’ll handle topic overlap, first-come preference, a direct comparison round, diversity of perspective as a tiebreaker, avoids an awkward last-minute call made under time pressure, and gives reviewers a consistent standard to apply rather than an ad hoc judgment each time it comes up.
- Weigh past speaking experience carefully, it’s a legitimate signal but shouldn’t be the whole picture. Past experience does predict some things, comfort on stage, ability to manage time, but requiring it exclusively can shut out strong first-time speakers with genuinely valuable expertise or perspective who simply haven’t had the opportunity yet. Consider a separate track or quota for first-time speakers if broadening your speaker pool is a program goal.
- Evaluate the proposal, not just the proposer’s resume. A well-known expert with a vague, unfocused proposal shouldn’t automatically outscore a less established applicant with a sharp, well-structured session outline. Keep proposal quality (clarity, structure, audience relevance) as its own explicit rubric line, separate from the applicant’s general credentials.
Recap
- Criteria typically include topic relevance, expertise, and presentation ability, weighed against limited available slots.
- Decide how to handle topic overlap between strong candidates in advance, not in the moment.
- Weigh past speaking experience carefully, requiring it exclusively can shut out strong first-time speakers.
- Score proposal quality as its own criterion, separate from the applicant’s general credentials or reputation.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What should be decided in advance regarding topic overlap between candidates?
2.Requiring past speaking experience exclusively has no real downside
3.Why score proposal quality separately from an applicants credentials?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- What happens if a selected speaker withdraws after acceptance? Keep a ranked waitlist from your original review, so a replacement can be contacted quickly rather than scrambling to find and vet someone new on short notice. This is common enough (schedule conflicts, changed circumstances) that it shouldn’t catch a well-planned program off guard.
- Can a speaker submit more than one talk proposal? Often yes, check the program’s specific rules, some programs cap it at one submission per person to keep the review process manageable and give more people a chance at a limited number of slots, others welcome multiple proposals and let the strongest one rise to the top.
- What if a speaker wants to make significant changes to their session after being accepted? Decide how much flexibility you’ll allow, minor adjustments are normal and expected, but a substantially different session than what was reviewed and accepted raises a fair question about whether it would have been accepted in its new form. Have a lightweight review step for material changes rather than assuming anything goes once someone’s accepted.
Recap
- Keep a ranked waitlist in case of speaker withdrawal, this is common enough to plan for.
- Multiple proposals per speaker depend on program-specific rules, state the policy clearly.
- Have a lightweight review process for significant session changes after acceptance.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What helps a program respond quickly if a speaker withdraws?
2.Every speaker program allows unlimited talk proposals per person.
3.What should happen if an accepted speaker wants to substantially change their session?
Part 9: Tenders
01
Planning a Tender Process
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Tenders are procurement processes, so planning centers on precise requirements and a documented, defensible evaluation process, since the outcome involves a real contract, real money, and often real scrutiny from unsuccessful bidders.
- Write requirements with real precision. Ambiguous requirements produce bids that are impossible to compare fairly against each other, one bidder interprets a vague spec generously, another interprets it conservatively, and you end up comparing apples to oranges through no fault of either bidder. The time spent tightening requirements before publishing a tender pays for itself many times over during evaluation.
- Document your evaluation process before bids come in, and stick to it. Tenders tend to draw more scrutiny than other program types if the process isn’t well documented, an unsuccessful bidder who’s spent real time and money on a proposal has a legitimate interest in understanding how the decision was made. A documented, consistently applied process is your best protection if a decision is ever questioned, and it’s simply the right way to run a fair process regardless.
- Decide your question-and-clarification process for bidders in advance. Bidders will have questions about ambiguous requirements, decide whether you’ll answer these individually or publish all questions and answers to every bidder (the fairer, more common approach in formal procurement), and set a firm cutoff date for questions well before the submission deadline.
Recap
- Tenders require precise requirements so bids can be fairly compared against each other.
- Document the evaluation process before bids arrive, and apply it consistently, not after the fact.
- Tenders tend to draw more scrutiny than other program types if the process isn’t well documented.
- Decide and publish your question-and-clarification process for bidders in advance.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Why does documenting the evaluation process matter especially for tenders?
2.Ambiguous requirements make bids easier to compare fairly
3.What is a fairer approach to bidder questions, common in formal procurement?
02
Judging Criteria for Tenders
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Tender evaluation typically weighs cost, capability, and compliance with requirements, usually with explicit numeric weighting since the process needs to be defensible after the fact, to bidders, to leadership, and sometimes to auditors.
- Cost alone rarely wins a well-run tender, and treating it as the deciding factor is a common, costly mistake. The lowest bid that can’t actually deliver, on time, to spec, or at all, ends up costing far more than a well-vetted, slightly higher bid from a capable provider. Weight cost as one factor among several, not the tiebreaker that overrides everything else.
- Compliance with stated requirements is often a pass/fail gate applied before scoring even begins, not a scored criterion itself. Non-compliant bids, missing a mandatory certification, failing to meet a stated technical requirement, typically get excluded from scoring entirely rather than scored low and compared against compliant bids. Make this gate explicit in your evaluation documentation so it’s clear why a bid was excluded rather than simply scored poorly.
- Build in a capability-verification step beyond what’s written in the proposal itself. A strong written proposal doesn’t guarantee actual delivery capability. Reference checks, past performance review, or a capability presentation from finalist bidders catches gaps that a paper proposal alone won’t reveal.
Recap
- Criteria typically weigh cost, capability, and compliance, with explicit numeric weighting for a defensible process.
- The lowest-cost bid isn’t automatically the best choice, weight cost as one factor, not the deciding one.
- Non-compliant bids are usually excluded before scoring begins, as a pass/fail gate, not scored low.
- Verify capability beyond the written proposal, through references or past performance, before finalizing a decision.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What commonly happens with a bid that does not meet stated requirements?
2.The lowest-cost bid should always win a well-run tender.
3.What helps verify capability beyond a written proposal?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- Can a bidder submit two competing proposals for the same tender? Usually not, most tenders require one proposal per bidding entity to keep comparison fair and prevent a single bidder from effectively taking up multiple slots in a limited evaluation process. State this explicitly in your tender documentation.
- What happens if a winning bid later can’t be honored? Most well-run tenders name a fallback in advance, commonly moving to the next-ranked compliant bid rather than reopening the entire process, which saves significant time and cost. Decide and document this fallback before you ever announce a winner.
- Can a bidder appeal or challenge an evaluation decision? Decide your policy on this before the tender opens, not after a challenge arrives. Many formal procurement processes build in a structured appeals or debrief process, offering unsuccessful bidders a documented explanation of the decision, both as a matter of fairness and as protection against a decision being challenged without a clear record to point to.
Recap
- One proposal per bidder is the standard rule, state it explicitly.
- Define a fallback for an unhonored winning bid in advance, commonly the next-ranked compliant bid.
- Decide your appeals or debrief policy before the tender opens, not after a challenge arrives.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Can a bidder usually submit two competing proposals to the same tender?
2.Tenders typically have no fallback plan if a winning bid falls through
3.When should an appeals or debrief policy be decided?
Part 10: Voting
01
Planning a Public Voting Program
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Voting-based programs put some or all of the decision in the public's hands, so planning has to account for fraud prevention and clear communication about how much the vote actually counts, since public trust in the result depends on both.
- Decide upfront how much weight public voting carries in the final result, and state that clearly to entrants and voters from the start. Entirely, partially alongside judge scores, or purely promotional with no actual bearing on the outcome, these are very different programs even if they look similar on the surface. Ambiguity here erodes trust fast, if voters believe their vote decides the winner and it turns out to be one input among many, that’s a credibility problem you created by not being upfront.
- Plan for basic safeguards against repeat voting from the very start of design, not as a patch after you notice a problem. Credibility depends on the result reflecting genuine, distinct participation, not on raw vote totals that might include the same handful of people voting repeatedly. Decide your verification approach (email confirmation, login requirement, one vote per device) based on how much is riding on the outcome.
- Think through how you’ll communicate voting progress, if at all, while voting is still open. Showing a live leaderboard can drive engagement and excitement, but it can also create bandwagon effects or discourage voters once an entry appears to have an insurmountable lead. Decide deliberately whether real-time visibility serves your program’s actual goals.
Recap
- Decide and clearly state how much weight voting carries in the final result, from the very start.
- Plan repeat-voting safeguards into the initial design, not as a later patch.
- Deliberately decide whether to show live voting progress, it affects voter behavior in both directions.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What should be clearly communicated to entrants and voters from the start?
2.Repeat-voting safeguards are optional for a credible voting program
3.What is a consideration around showing a live leaderboard during voting?
02
Judging Criteria for Voting
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Where voting is combined with expert judging, the two need a clearly defined formula for how they combine, decided before either process starts, not calculated after the fact once both sets of results are sitting in front of you.
- A common approach blends a percentage of the public vote with a percentage of judge scores, but the exact split should reflect what the program actually values. A 50/50 split treats popularity and expert assessment as equally important, a 20/80 split toward judges treats voting as a minor input. Neither is wrong, but the choice should be a deliberate reflection of your program’s actual goals, not an arbitrary default.
- Whatever the formula, publish it before voting opens, not after. Calculating a blend after the fact invites doubt about the result regardless of how fairly it was actually done, once people can see that a formula was applied after votes were already in, it’s reasonable for them to wonder whether it was chosen to produce a particular outcome, even if that’s not what happened. Publishing the formula upfront removes that doubt entirely.
- Decide how you’ll normalize public vote counts against judge scores, which are usually on very different scales. A raw vote count of thousands and a judge score out of 100 aren’t directly combinable without some normalization step, converting both to a percentage of the maximum possible, for instance. Work this out and document it before you’re staring at two very different-looking sets of numbers trying to combine them.
Recap
- Define exactly how voting and judge scores combine before either process starts, not after.
- The split should reflect what the program actually values, not default to an arbitrary number.
- Publish the formula before voting opens to avoid doubt later, regardless of how fairly it was actually applied.
- Decide how you’ll normalize vote counts against judge scores, which are usually on very different scales.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.When should the formula for combining votes and judge scores be published?
2.Calculating the blend formula after the fact tends to invite doubt about the result.
3.Why does normalization matter when combining votes and judge scores
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- Can the same public voter vote more than once? Most well-run programs prevent this through email or login verification, though the strength of verification should scale with what’s riding on the outcome, a casual engagement contest needs less rigor than a program with a meaningful prize attached. Check and state the specific program’s safeguards clearly.
- What happens if voting numbers look suspicious? Most credible programs reserve the right to investigate and exclude suspect votes, a sudden, implausible spike in votes for one entry, votes from clearly duplicated accounts, and this right should be stated clearly in the rules in advance, not invoked as a surprise after the fact.
- Can voting be paused or extended if there’s a technical problem during the voting window? Decide this in advance rather than improvising, a clear stated policy (for example, extending the window by exactly the duration of any verified outage) protects both the program’s credibility and prevents accusations of favoring one outcome by choosing to extend or not extend selectively.
Recap
- Repeat voting is normally prevented through verification, scaled to what’s riding on the outcome.
- Programs typically reserve the right to investigate and exclude suspect votes, stated clearly in advance.
- Decide your policy on pausing or extending voting for technical issues in advance, not improvised in the moment.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What commonly prevents the same voter from voting multiple times?
2.Most voting programs have no recourse if voting numbers look suspicious.
3.When should a policy on extending voting for technical issues be decided?
Part 11: Creative
01
Planning a Creative Program
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Creative programs judge subjective work, art, design, writing, music, so planning needs to account for genuinely varied submission formats and criteria that stay meaningful despite that subjectivity, a harder design problem than it might first appear.
- Decide what file types and formats you’ll accept early, and be specific. Creative submissions vary more in format than almost any other program type in this book, a photograph, a video, an audio file, a physical piece represented only through images, and vague rules here create real headaches, both for entrants trying to figure out how to submit and for judges trying to fairly compare wildly different formats. Be as specific as the medium allows.
- Build a rubric specifically designed for subjective work, since “I know good work when I see it” doesn’t hold up as a shared standard across multiple judges. This is harder than it sounds, but breaking an overall impression into named sub-criteria, even for something as subjective as visual art or music, gives judges a shared vocabulary and produces more consistent, more defensible scoring than an unstructured gut reaction.
- Think through how entries spanning very different styles or mediums within one category will be compared fairly. A minimalist piece and a maximalist piece in the same “Best Design” category aren’t directly comparable on style, but they can be compared on execution, originality within their own approach, and how well they achieve what they’re going for. Make sure your rubric supports that kind of comparison rather than implicitly rewarding one aesthetic over another.
Recap
- Creative submissions vary widely in format, define accepted types early and be specific.
- Build a rubric that works for subjective judgment, breaking overall impression into named sub-criteria.
- Design your rubric so entries in very different styles within one category can still be compared fairly.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.What creates real headaches in creative programs if left vague?
2."I know good work when I see it" is a reliable shared standard across judges.
3.How should very different styles within one category be compared fairly?
02
Judging Criteria for Creative Work
Four recurring motivations — and why the dominant one shapes every decision downstream.
5–8 min read
In plain terms: Creative criteria typically weigh originality, execution, and impact, structured deliberately to reduce the risk of judging becoming pure personal taste, which is the single biggest threat to credibility in this program type.
- Breaking “quality” into specific sub-criteria gives judges something concrete to anchor to. Technique, originality, coherence, impact, named separately, rather than one overall impression that varies wildly between judges depending on personal taste. This doesn’t eliminate subjectivity, creative judgment is inherently subjective, but it channels that subjectivity into more consistent, comparable, and explainable scores.
- Where entries span very different mediums or styles within one category, judges should be told explicitly not to penalize an entry simply for being different from what they personally prefer. This needs to be stated directly in judge briefing materials, not assumed. Judges bring genuine personal taste to creative work, that’s part of what makes them qualified, but personal taste needs to be separated from the rubric’s actual criteria during scoring.
- Consider a calibration step before real scoring begins, even a short one. Having judges independently score the same two or three sample entries first, then briefly discuss the range of scores given, surfaces very different interpretations of the rubric before it affects a full round of real entries. This matters more in creative judging than almost anywhere else in this book, because subjective interpretation gaps between judges tend to be widest here.
Recap
- Criteria typically include originality, execution, and impact, broken into specific, named sub-criteria.
- Breaking quality into sub-criteria channels subjectivity into more consistent, explainable scoring.
- Explicitly instruct judges not to penalize entries for differing from personal taste, state this directly.
- Consider a short calibration step before real scoring, interpretation gaps tend to be widest in creative judging.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Why break "quality" into specific sub-criteria for creative judging?
2.Judges should feel free to score down entries that simply differ from their personal taste
3.Why does calibration matter especially in creative judging?
03
Common Questions
The eight core jobs every platform in this category is built around.
5–8 min read
- Can the same piece of creative work be entered in two different categories? Usually not, most programs require one category per piece to keep judging fair and comparisons meaningful, allowing the same piece to compete in multiple categories can also feel, to other entrants, like an unfair attempt to maximize a single strong piece’s odds of winning something.
- What if a judge has a strong personal style preference that clashes with an entry? Ground scoring in the rubric’s specific criteria rather than overall taste, that’s exactly what a well-built rubric is there to prevent. If a judge finds they genuinely cannot separate personal taste from fair evaluation of a specific entry, treat it similarly to a conflict of interest, better to flag it and have another judge weigh in than force a score that isn’t really evaluating the work.
- What if an entry appears to closely resemble another artist’s or creator’s existing, unrelated work? Decide your policy on originality concerns in advance, most programs distinguish between clear plagiarism (which should disqualify an entry) and coincidental similarity or shared influence within a genre or style (which shouldn’t). Give judges a way to flag a concern for review rather than making a unilateral disqualification decision on the spot.
Recap
- One category per piece is the standard rule, both for fair comparison and fairness to other entrants.
- Rubric criteria exist precisely to keep personal taste from driving the score, treat an inability to separate the two like a conflict of interest.
- Decide your originality/plagiarism policy in advance, and give judges a flagging process rather than unilateral disqualification power.
Quick check
Regardless of what a program is called, the design questions behind it are mostly the same:
1.Can the same creative piece usually be entered in two categories
2.Personal style preference should be allowed to override rubric-based scoring





























