![]() ![]() ![]() ![]() Has the 30 problems completed by our users in a user study. Has a dozen puzzles indicating which ones the AI solved and did not Try the notebook at BinderĪnd see how your programming compares to the AI baselines! The notebooks subdirectory has some relevant notebooks. (NEW!) Problems inspired by the human-eval dataset from the codex paper.Olympiad problems from the International Collegiate Programming Contest and International Mathematical Olympiad.The website, a popular website for programming competition problems.Wikipedia articles about algorithms, puzzles,.2021 (Link to be added shortly) Click here to browse the puzzles and solutions Programming Puzzles, by Tal Schuster, Ashwin Kalyan, Alex Polozov, and Adam Tauman Kalai. If it solved the puzzle, then it succeeded by definition.įor more information on the motivation and how programming puzzles can help AI learn to program, see Have the advantage that they are unambiguous, there is no need to debug the AI-generated code or fears that itĭoesn't do what you want. So what is the use there? Code-based specs Specs are notoriously ambiguous and test the system's understanding of English.Īnd with input-output test cases, you would have to have solved a puzzle before you pose it, Other datasets usually use a combination of English and a hidden test set of input-output pairs. In puzzles, the spec is defined by code, while Many programming datasets have been proposed over the years, and several have problems of a similar nature Then we could make progress on some important algorithms problems.īut until then, a second reason is that they can be valuable for training and evaluating AI systems. Why puzzles? One reason is that, if we can solve them better than human programmers, intersection( N)) = ( 1 if j in N else 2) for i in range( 99) for j in range( i)) ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |