Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Small bug fixes for running models without tokenizers #168

Merged
merged 5 commits into from
Aug 28, 2024
Merged

Small bug fixes for running models without tokenizers #168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
43258fe
First pass on v1 model
Landanjs Jun 11, 2024
0e0fe80
Update train
corystephenson-db Aug 27, 2024
e03fe0c
model_max_length
corystephenson-db Aug 27, 2024
7f97fd6
use right tokenizer
corystephenson-db Aug 27, 2024
8eff9a2
no model class yet
corystephenson-db Aug 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions diffusion/callbacks/log_diffusion_images.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def __init__(self,
latent_batch = {}
tokenized_t5 = t5_tokenizer(batch,
padding='max_length',
max_length=t5_tokenizer.model.max_length,
max_length=t5_tokenizer.model_max_length,
truncation=True,
return_tensors='pt')
t5_attention_mask = tokenized_t5['attention_mask'].to(torch.bool).cuda()
Expand All @@ -108,7 +108,7 @@ def __init__(self,

tokenized_clip = clip_tokenizer(batch,
padding='max_length',
max_length=t5_tokenizer.model.max_length,
max_length=clip_tokenizer.model_max_length,
truncation=True,
return_tensors='pt')
clip_attention_mask = tokenized_clip['attention_mask'].cuda()
Expand Down
9 changes: 6 additions & 3 deletions diffusion/train.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,17 +88,20 @@ def train(config: DictConfig) -> None:

model: ComposerModel = hydra.utils.instantiate(config.model)

# If the model has a tokenizer, we'll need it for the dataset
if hasattr(model, 'tokenizer'):
tokenizer = model.tokenizer
else:
tokenizer = None

if hasattr(model, 'autoencoder_loss'):
# Check if this is training an autoencoder. If so, the optimizer needs different param groups
optimizer = make_autoencoder_optimizer(config, model)
tokenizer = None
elif isinstance(model, ComposerTextToImageMMDiT):
# Check if this is training a transformer. If so, the optimizer needs different param groups
optimizer = make_transformer_optimizer(config, model)
tokenizer = model.tokenizer
else:
optimizer = hydra.utils.instantiate(config.optimizer, params=model.parameters())
tokenizer = model.tokenizer

# Load train dataset. Currently this expects to load according to the datasetHparam method.
# This means adding external datasets is currently not super easy. Will refactor or check for
Expand Down
Loading