odilf/cse2000-software-project
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: Polished READMEs and CONTRIBUTINGs

Browse source
This commit is contained in:
Ody Machairas 2024-05-29 00:17:18 +02:00
parent e725fe6bf9
commit d0cf77402a
8 changed files with 264 additions and 173 deletions
Download patch file Download diff file Expand all files Collapse all files

40
CONTRIBUTING.md
View file

@ -20,32 +20,34 @@ Install the [pre-commit hook](https://pre-commit.com/#install). For example:
```bash
pip install pre-commit
pre-commit install
```
These hooks use ruff to format the Python kernel code, and pyright to lint it. For svelte we use ESLint and Prettier.
### conventional-pre-commit
To use the conventional-pre-commit, which enforces [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/), run the following command:
```bash
pre-commit install --hook-type commit-msg
```
These hooks mainly run formatters and linters for Python and Svelte. They also enforce [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). In general, this project uses the conventional commits convention, so we expect all commits to follow it (except merge commits). If a PR does not follow conventional commits it will be squashed.
## Making code changes
Now make your changes. Make sure to include additional tests if necessary.
Spectral is divided in two parts, the app (SvelteKit) and the kernel (Python). You can refer to the readmes of the [app](app/README.md) and [kernel](/kernel/README.md) respectively to gain more insight into how to contribute.
To try out your changes, you can run the whole application using docker. First install [docker](https://docs.docker.com/engine/install/) and [docker compose](https://docs.docker.com/compose/install/), and run:
```bash
docker compose up --build
```
Make sure to add tests when appropriate. Ideally, there should be a test that would fail without your PR. The `ci.py` script does all testing, typechecking and linting:
```bash
./ci.py
```
Verify that the tests are passing! The CI will run the tests, but you can might want to run them locally as well. You can run only integration tests by doing:
```bash
???
./ci.py integration
```
Also, feel free to run the whole server locally with:
```bash
docker-compose up --build
```
Please make sure to keep a reasonable git history following conventional commits. If this is not the case in a PR, the PR will be squashed by the maintainers.
After you are done with the feature, you can push your changes back to your fork:
@ -53,7 +55,7 @@ After you are done with the feature, you can push your changes back to your fork
git push
```
Finally, open a pull request on GitHub (or GitLab)!
Finally, open a pull request on GitHub!
## Making documentation changes
@ -66,10 +68,14 @@ mkdocs serve
Then doing `Control-C`, and running:
```
```bash
mkdocs serve
```
(So you run `mkdocs serve` twice.)
You can then see your local copy of the documentation by navigating to `localhost:8000` in a web browser.
## Documenting code
As part of our standard, we require that non-trivial code to be explained with comments. Make sure the comments are reasonably written with your judgement.

145
README.md
View file

@ -1,93 +1,82 @@
# atypical-speech-project
# Spectral
TODO: Add badges
Spectral is a toolkit that enables comparative analysis of atypical speech, with a nice user experience.
## Getting started
It is designed to be modular and extensible.
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
<!--toc:start-->
- [Usage](#usage)
- [Running locally](#running-locally)
- [Self hosting](#self-hosting)
- [Contributing](#contributing)
- [License](#license)
<!--toc:end-->
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
## Add your files
- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
```
cd existing_repo
git remote add origin https://gitlab.ewi.tudelft.nl/cse2000-software-project/2023-2024/cluster-n/11c/atypical-speech-project.git
git branch -M main
git push -uf origin main
```
## Integrate with your tools
- [ ] [Set up project integrations](https://gitlab.ewi.tudelft.nl/cse2000-software-project/2023-2024/cluster-n/11c/atypical-speech-project/-/settings/integrations)
## Collaborate with your team
- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
- [ ] [Set auto-merge](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
## Test and Deploy
Use the built-in continuous integration in GitLab.
- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
***
# Editing this README
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thanks to [makeareadme.com](https://www.makeareadme.com/) for this template.
## Suggestions for a good README
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
## Name
Choose a self-explaining name for your project.
## Description
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
## Badges
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
## Visuals
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
## Installation
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
<img src="./docs/images/home.jpeg" alt="Home page of Spectral">
<img src="./docs/images/waveform.jpeg" alt="Waveform mode of Spectral">
## Usage
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
## Support
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
The easiest way to use the app is by accessing at https://spectral.ewi.tudelft.nl.
## Roadmap
If you have ideas for releases in the future, it is a good idea to list them in the README.
### Running locally
First, clone the repo:
```bash
git clone https://github/TBD/spectral.git
cd spectral
```
Download [Docker](TODO) and [Docker compose](TODO) following the instructions on the website.
(NOTE: For MacOS (especially Apple Sillicon), consider using [colima](TODO) as the runtime)
Then, run the compose file
```
docker compose up --build
```
This will pull the containers and install all dependencies. (NOTE: There is one Python dependency where wheels have to be rebuilt. This will likely take 5-10 minutes at least).
When all the containers are pulled and the dependencies built, the app should be available at `localhost` (i.e., `http://localhost:80`).
### Self hosting
Spectral can be easily self-hosted on your own server.
Follow the steps above and expose port 80. You could also change the nginx config to host the app in a different port. That is, in `nginx.conf` change the "listen 80;" line to your desired port:
```diff
...
server {
- listen 80;
+ listen <YOUR-PORT>;
...
}
...
```
<details>
<summary> If you really don't want two reverse proxies... </summary>
For more advanced users, if you are already using a reverse proxy then you can follow the configuration in the docker compose file and set up each service in your configuration.
For example, if you're already using compose then you can add the services with barely any changes to your existing compose file. This method as-explained is somewhat dubious fundamentally, but an advanced user can figure it out. Again, this is not really recommended nor officially supported by the project.
</details>
## Contributing
State if you are open to contributions and what your requirements are for accepting them.
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.
Contributions are welcomed. For more information, see the [CONTRIBUTING.md](/CONTRIBUTING.md)
## License
For open source projects, say how it is licensed.
## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
TODO

211
app/CONTRIBUTING.md
View file

@ -1,99 +1,178 @@
# Contributing App
# Contributing to app
Contributions (pull requests) to the app are very welcome! Here's how to get started.
Make sure to take a look to the [general contributing guidelines](../CONTRIBUTING.md).
## Getting started
There are two main aspects one might want to contribute to in this repo: adding a new mode or general contributing. Each one has different considerations, both of which are outlined below.
First fork the library on GitHub (or GitLab).
<!--toc:start-->
Then clone the library and go to the app folder.
- [General contributing](#general-contributing)
- [Framework](#framework)
- [Package manager](#package-manager)
- [Styling](#styling)
- [Adding a new mode](#adding-a-new-mode) - [Declaring mode data](#declaring-mode-data) - [Adding components](#adding-components) - [Registering the new mode](#registering-the-new-mode) - [Kernel side](#kernel-side)
<!--toc:end-->
<!-- TODO: change it to `app` when the time comes -->
## General contributing
### Framework
The framework used is SvelteKit. Svelte and SvelteKit are some of the quickest-to-learn frameworks out there. The best way to do is by following [the official tutorial](https://learn.svelte.dev).
### Package manager
We use pnpm. Install pnpm following [the instructions on the website](https://pnpm.io/installation).
Then install dependencies with
```bash
git clone https://github.com/your-username-here/spectral.git
cd spectral/app
pnpm install # or `pnpm i`
```
## Contributing
Dependencies should be automatically installed with `docker compose up --build`. However, you should install dependencies for the IDE language server to pick up type definitions and such.
As stated in the root folder's CONTRIBUTING.md, please use the general guidelines that we agreed upon. This includes:
Other than that, `pnpm` is mostly used to add new dependencies (either `pnpm add <dep>` for production dependencies or `pnpm add -D <dep>` for development dependencies)
### pre-commit
### Styling
We use [tailwind](TODO) for styling and [shadcn-svelte](https://www.shadcn-svelte.com/docs) for components.
Try to use tailwind for most styling. Of course, if you want to something more complicated you can use regular `<style>` tags.
You can browse the available shadcn components and usage at [the website](https://www.shadcn-svelte.com/docs). If a component is not in the project already, you can add it via:
```bash
pip install pre-commit
pre-commit install
pnpm dlx shadcn-svele add <component>
```
#### formatting hook
## Adding a new mode
The pre-commit hooks enforce formatting with prettier and ESLint with each commit. It is equivalent to doing:
Adding new modes of analysis is designed to be as straghtforward as possible, while still being as flexible as possible.
```bash
pnpm prettier --write . && pnpm eslint --fix .
As an overview, each modes needs to declare:
- A name
- Data it expects from the kernel
- File state
- Mode state
- A component that displays the data
- An icon
> If any step in the following instructions is unclear, you can always take a look to how other modes are implemented.
### Declaring mode data
First, create the file `app/src/lib/analysis/modes/<your-mode>/index.ts`. Here you declare the data following this template:
```typescript
import { z } from 'zod';
import type { ModeValidator } from '..';
import { fileState } from '../file-state';
export const yourModeData = {
computedFileData: /* TODO */,
fileState: fileState
.pick({
/* TODO */
})
.default({}),
modeState: z.object({
/* TODO */
}).default({})
} satisfies ModeValidator;
```
This is a "custom" hook that we created and as such does not require downloading any package.
Then you need to fill in each field. Each field is a [zod](TODO) parser, which defines shapes that can be validated at runtime. Visit the [zod docs](TODO) for more info.
#### conventional-pre-commit
- `computedFileData` is the data that is directly computed from the file. This info should be implemented in the Kernel.
- `modeState` is synced state that is stored per-mode (and per-pane). Used for things such as toggles and sliders set by the user. For example, whether to show the legend in the graph.
- `fileState` is state that is stored with each file, globally. For example, playback speed or selected frame. The file state is global for all modes. So, instead of declaring it in the mode data directly, it is declared in `file-state` and the mode picks fields from there. The file state is passed to the kernel as additional info (this is mainly used to send frames).
To use the conventional-pre-commit, which enforces [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/), run the following command:
Both `modeState` and `fileState` need to provide a default (this is because the state needs to be somehow initialized at some point).
```bash
pre-commit install --hook-type commit-msg
```
### Adding components
## Making code changes
Now make your changes. Make sure to include additional tests if necessary.
### Running the application
While making code changes, it might be required to be in contact with another service other than app. To simulate this, we provide a docker-compose file in the root directory of this project. To run the docker services, go to _project's root folder_ and write:
```bash
docker compose up --build
```
The app service should be accessible from `http://localhost`.
### Before committing!!
Before committing, make sure all tests are passing (this includes unit and integration tests). To run all tests, execute the following command in the app folder:
```bash
pnpm test
```
### Documentation
As part of our standard, we require that non-trivial code to be explained with comments. Make sure the comments are reasonably written with your judgement.
### Adding a new mode
In order to add a new mode for all users to be able to access, there are several steps required. At every step, you can look at how other modes are implemented in case there's any confusion.
First, create a folder `src/lib/analysis/modes/{your-mode}/`. Create an `index.ts` inside that exports a zod validator object with the necessary data to render that mode. Then, create a new component in that folder called `YourMode.svelte`, that takes in a `data` prop, as so:
Create a new component in `your-mode/YourMode.svelte`:
```svelte
<script lang="ts">
export let data: SpecificModeData<'your-mode'>[];
import type { ModeComponentProps } from '..';
export let fileData: ModeComponentProps<'your-mode'>['fileData'];
export let modeState: ModeComponentProps<'your-mode'>['modeState'];
</script>
<!-- Implement your component here -->
```
It is convenient to re-export the svelte component from `index.ts`.
It is convenient to re-export the new mode from `.../modes/<your-mode>/index.ts`:
Then you need to register the mode. You do this in `src/lib/analysis/modes/index.ts`. Add your zod validator to the union named `modeDataValidator`. Add the name of the mode to the `modes` string array. Finally, return the corresponding svelte component in `getComponent` and an appropriate icon in `getIcon`.
```ts
/** ... */
The Python server will need to be updated to have an appropriate endpoint for that mode, that returns the specified data.
### Finishing off
After you are done with the feature, and made sure to follow what we said previously, you can push your changes back to your fork:
```bash
git push
export { default as YourMode } from 'YourMode.svelte';
```
Finally, open a pull request on GitHub (or GitLab)!
### Registering the new mode
Finally you have to register your new mode in `modes/index.ts`. You need to add the mode data in the `modes` constant:
```ts
export const modes = {
'simple-info': simpleInfoData,
waveform: waveformData,
spectrogram: spectrogramData,
'vowel-space': vowelSpaceData,
transcription: transcriptionData,
'your-mode': yourModeData
} as const satisfies Record<string, ModeValidator>;
```
Then register the component and the icon
```ts
export const modeComponents: {
[M in keyof typeof modes]: {
component: ModeComponent<M>;
icon: ComponentType;
};
} = {
'simple-info': {
component: SimpleInfo,
icon: InfoIcon
},
waveform: {
component: Waveform,
icon: AudioWaveformIcon
},
spectrogram: {
component: Spectrogram,
icon: SpectrogramIcon
},
'vowel-space': {
component: VowelSpace,
icon: VowelSpaceIcon
},
transcription: {
component: Transcription,
icon: CaptionsIcon
},
'your-mode': {
component: YourMode,
icon: YourModeIcon
}
};
```
The icon can be custom made, but you can also use one from [lucide](https://lucide.dev/) if appropriate.
### Kernel side
You need to implement and endpoint in the Python side with the same name as your mode that provides whatever you have declared that `computedData` should return.

36
app/README.md
View file

@ -1,10 +1,32 @@
app for the application, using [SvelteKit](https://kit.svelte.dev/).
App side of Spectral, using [SvelteKit](https://kit.svelte.dev/).
## Developing
Make sure to also read the [general README](../README.md).
We use pnpm. Install pnpm following [the instructions on the website](https://pnpm.io/installation).
---
Then install dependencies with `pnpm install` (or `pnpm i`). Run the dev server with
The app is responsible not only for the UI, but also for most UX-related aspects.
That is, the app handles:
- General user interface
- Account system, login, etc.
- Managing the data in the db
- Storing state for sessions, modes and files
- Making requests to the kernel
However, the app is mostly responsible for providing an interface to easily create new modes of analysis, as components that display/visualize data fetched from the kernel.
## Contributing
For more info about how to contribute please refer to the CONTRIBUTING.md files:
- [General](./CONTRIBUTING.md)
- [App](./CONTRIBUTING.md)
- [Kernel](../kernel/CONTRIBUTING.md)
## Running standalone
The recommended way to run the app is with docker compose. However, if you want to run the app side by itself, you can do that by installing dependencies with `pnpm install` (or `pnpm i`) and running the dev server with
```bash
pnpm dev
@ -13,11 +35,7 @@ pnpm dev
pnpm dev -- --open
```
### Adding components
We use [shadcn-svelte](https://www.shadcn-svelte.com/docs) to add components. Search for a component there and run the appropriate command to install the component.
## Building
### Building
Create a production version of the app:

1
app/src/routes/(protected)/db/file/[fileId]/+server.ts
View file

@ -28,7 +28,6 @@ async function fetchName(fileId: string, userId: string): Promise<Response> {
}
async function fetchData(fileId: string, userId: string): Promise<Response> {
console.log({ userId });
try {
const row = await db.query.fileTable.findFirst({
columns: {

BIN
docs/images/home.jpeg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 144 KiB

BIN
docs/images/waveform.jpeg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 340 KiB

4
kernel/README.md
View file

@ -2,10 +2,10 @@
## Run kernel locally
For the kernel we are using fastapi. To start up the server locally run:
For the kernel we are using uvicorn. To start up the server locally run:
```bash
fastapi dev spectral/main.py
uvicorn dev spectral/main.py
```
This will allow you to look at the autogenerated OpenAPI documentation at http://127.0.0.1:8000/docs and make calls to the api via http://127.0.0.1:8000/