| chapter_0.rs |
# Chapter 0: Introduction
This tutorial assumes that you are:
- Already familiar with Rust
- Using `winnow` for the first time
The focus will be on parsing in-memory strings (`&str`). Once done, you might want to check the
[Special Topics][_topic] for more specialized topics or examples.
## About
`winnow` is a parser-combinator library. In other words, it gives you tools to define:
- "parsers", or functions that take an input and give back an output
- "combinators", or functions that take parsers and _combine_ them together!
While "combinator" might be an unfamiliar word, you are likely using them in your rust code
today, like with the [`Iterator`] trait:
```rust
let data = vec![1, 2, 3, 4, 5];
let even_count = data.iter()
.copied() // combinator
.filter(|d| d % 2 == 0) // combinator
.count(); // combinator
```
Parser combinators are great because:
- Individual parser functions are small, focused on one thing, ignoring the rest
- You can write tests focused on individual parsers (unit tests and property-based tests)
in addition to testing the top-level parser as a whole.
- Top-level parsing code looks close to the grammar you would have written |
1457 |
| chapter_1.rs |
# Chapter 1: The Winnow Way
First of all, we need to understand the way that winnow thinks about parsing.
As discussed in the introduction, winnow lets us compose more complex parsers from more simple
ones (using "combinators").
Let's discuss what a "parser" actually does. A parser takes an input and advances it until it returns
a result, where:
- `Ok` indicates the parser successfully found what it was looking for; or
- `Err` indicates the parser could not find what it was looking for.
Parsers do more than just return a binary "success"/"failure" code.
- On success, the parser will return the processed data. The input will be advanced to the end of
what was processed, pointing to what will be parsed next.
- If the parser failed, then there are multiple errors that could be returned.
We'll explore this further in [`chapter_7`].
```text
┌─► Ok(what matched the parser)
┌────────┐ │
my input───►│a parser├──►either──┤
└────────┘ └─► Err(...)
```
To represent this model of the world, winnow uses the [`Result<O>`] type.
The `Ok` variant has `output: O`;
whereas the `Err` variant stores an error.
You can import that from:
```rust
use winnow::Result;
```
To combine parsers, we need a common way to refer to them which is where the [`Parser<I, O, E>`]
trait comes in with [`Parser::parse_next`] being the primary way to drive
parsing forward.
In [`chapter_6`], we'll cover how to integrate these into your application, particularly with
[`Parser::parse`].
You'll note that `I` and `O` are parameterized -- while most of the examples in this book
will be with `&str` (i.e. parsing a string); [they do not have to be strings][stream]; nor do they
have to be the same type (consider the simple example where `I = &str`, and `O = u64` -- this
parses a string into an unsigned integer.)
# Let's write our first parser!
The simplest parser we can write is one which successfully does nothing.
To make it easier to implement a [`Parser`], the trait is implemented for
functions of the form `Fn(&mut I) -> Result<O>`.
This parser function should take in a `&str`:
- Since it is supposed to succeed, we know it will return the `Ok` variant.
- Since it does nothing to our input, the input will be left where it started.
- Since it doesn't parse anything, it also should just return an empty string.
```rust
use winnow::Result;
use winnow::Parser;
pub fn do_nothing_parser<'s>(input: &mut &'s str) -> Result<&'s str> {
Ok("")
}
fn main() {
let mut input = "0x1a2b Hello";
let output = do_nothing_parser.parse_next(&mut input).unwrap();
// Same as:
// let output = do_nothing_parser(&mut input).unwrap();
assert_eq!(input, "0x1a2b Hello");
assert_eq!(output, "");
}
``` |
3410 |
| chapter_2.rs |
|
9100 |
| chapter_3.rs |
|
11270 |
| chapter_4.rs |
# Chapter 4: Parsers With Custom Return Types
So far, we have seen mostly functions that take an `&str`, and return a
[`Result<&str>`]. Splitting strings into smaller strings and characters is certainly
useful, but it's not the only thing winnow is capable of!
A useful operation when parsing is to convert between types; for example
parsing from `&str` to another primitive, like [`usize`].
All we need to do for our parser to return a different type is to change
the type parameter of [`Result`] to the desired return type.
For example, to return a `usize`, return a `Result<usize>`.
One winnow-native way of doing a type conversion is to use the
[`Parser::parse_to`] combinator
to convert from a successful parse to a particular type using [`FromStr`].
The following code converts from a string containing a number to `usize`:
```rust
# use winnow::prelude::*;
# use winnow::Result;
# use winnow::ascii::digit1;
#
fn parse_digits(input: &mut &str) -> Result<usize> {
digit1
.parse_to()
.parse_next(input)
}
fn main() {
let mut input = "1024 Hello";
let output = parse_digits.parse_next(&mut input).unwrap();
assert_eq!(input, " Hello");
assert_eq!(output, 1024);
assert!(parse_digits(&mut "Z").is_err());
}
```
`Parser::parse_to` is just a convenient form of [`Parser::try_map`] which we can use to handle
all radices of numbers:
```rust
# use winnow::prelude::*;
# use winnow::Result;
# use winnow::token::take_while;
use winnow::combinator::dispatch;
use winnow::token::take;
use winnow::combinator::fail;
fn parse_digits(input: &mut &str) -> Result<usize> {
dispatch!(take(2usize);
"0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
"0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
"0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
"0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
_ => fail,
).parse_next(input)
}
// ...
# fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='1'),
# )).parse_next(input)
# }
#
# fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='7'),
# )).parse_next(input)
# }
#
# fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='9'),
# )).parse_next(input)
# }
#
# fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='9'),
# ('A'..='F'),
# ('a'..='f'),
# )).parse_next(input)
# }
fn main() {
let mut input = "0x1a2b Hello";
let digits = parse_digits.parse_next(&mut input).unwrap();
assert_eq!(input, " Hello");
assert_eq!(digits, 0x1a2b);
assert!(parse_digits(&mut "ghiWorld").is_err());
}
```
See also [`Parser`] for more output-modifying parsers. |
3520 |
| chapter_5.rs |
|
9947 |
| chapter_6.rs |
# Chapter 6: Integrating the Parser
So far, we've highlighted how to incrementally parse, but how do we bring this all together
into our application?
Parsers we've been working with look like:
```rust
# use winnow::error::ContextError;
# use winnow::Parser;
use winnow::Result;
pub fn parser<'s>(input: &mut &'s str) -> Result<&'s str> {
// ...
# Ok("")
}
```
1. We have to decide what to do about the "remainder" of the `input`.
2. The [`Result`] may not be compatible with the rest of the Rust ecosystem.
Normally, Rust applications want errors that are `std::error::Error + Send + Sync + 'static`
meaning:
- They implement the [`std::error::Error`] trait
- They can be sent across threads
- They are safe to be referenced across threads
- They do not borrow
winnow provides [`Parser::parse`] to help with this:
- Ensures we hit [`eof`]
- Wraps the error in [`ParseError`]
- For simple cases, [`ParseError`] provides a [`std::fmt::Display`] impl to render the error.
- For more involved cases, [`ParseError`] provides the original [`input`][ParseError::input] and the
[`offset`][ParseError::offset] of where it failed so you can capture this information in
your error, [rendering it as you wish][chapter_7#error-adaptation-and-rendering].
- Converts from [`ModalResult`] to [`Result`] (if used, more on this in [`chapter_7`])
However, [`ParseError`] will still need some level of adaptation to integrate with your
application's error type (like with `?`).
```rust
# use winnow::prelude::*;
# use winnow::Result;
# use winnow::token::take_while;
# use winnow::combinator::dispatch;
# use winnow::token::take;
# use winnow::combinator::fail;
use winnow::Parser;
#[derive(Debug, PartialEq, Eq)]
pub struct Hex(usize);
impl std::str::FromStr for Hex {
type Err = anyhow::Error;
fn from_str(input: &str) -> Result<Self, Self::Err> {
parse_digits
.map(Hex)
.parse(input)
.map_err(|e| anyhow::format_err!("{e}"))
}
}
// ...
# fn parse_digits<'s>(input: &mut &'s str) -> Result<usize> {
# dispatch!(take(2usize);
# "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
# "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
# "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
# "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
# _ => fail,
# ).parse_next(input)
# }
#
# fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='1'),
# )).parse_next(input)
# }
#
# fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='7'),
# )).parse_next(input)
# }
#
# fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='9'),
# )).parse_next(input)
# }
#
# fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
# take_while(1.., (
# ('0'..='9'),
# ('A'..='F'),
# ('a'..='f'),
# )).parse_next(input)
# }
fn main() {
let input = "0x1a2b";
assert_eq!(input.parse::<Hex>().unwrap(), Hex(0x1a2b));
let input = "0x1a2b Hello";
assert!(input.parse::<Hex>().is_err());
let input = "ghiHello";
assert!(input.parse::<Hex>().is_err());
}
``` |
4103 |
| chapter_7.rs |
|
25897 |
| chapter_8.rs |
# Chapter 8: Debugging
When things inevitably go wrong, you can introspect the parsing state by running your test case
with `--features winnow/debug`.
For example, the trace output of an [escaped string parser][crate::_topic::language#escaped-strings]:
You can extend your own parsers to show up by wrapping their body with
[`trace`][crate::combinator::trace]. Going back to [`do_nothing_parser`][super::chapter_1].
```rust
# use winnow::ModalResult;
# use winnow::Parser;
use winnow::combinator::trace;
pub fn do_nothing_parser<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
trace(
"do_nothing_parser",
|i: &mut _| Ok("")
).parse_next(input)
}
#
# fn main() {
# let mut input = "0x1a2b Hello";
#
# let output = do_nothing_parser.parse_next(&mut input).unwrap();
# // Same as:
# // let output = do_nothing_parser(&mut input).unwrap();
#
# assert_eq!(input, "0x1a2b Hello");
# assert_eq!(output, "");
# }
``` |
1337 |
| mod.rs |
# Tutorial
Table of Contents |
252 |